2024-02-10T07:21:28.009 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-10T07:21:28.009 DEBUG:teuthology.run:Teuthology command: teuthology --description orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} --owner scheduled_yuriw@teuthology --name yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217 --verbose -- /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217/orig.config.yaml 2024-02-10T07:21:28.038 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T07:21:28.119 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217 branch: main description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} email: yweinste@redhat.com first_in_suite: false job_id: '7555217' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) mon_bind_msgr2: false sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: main sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 1610 sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 sleep_before_teardown: 0 subset: 111/120000 suite: orch suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6t5GCr8qlR/nF+e2IdSDCRLhpYRjYi9g1KBMBI9nehO22Aod6MfVunPPjNGq/ORhO4eH+cNSTIvWQJei7iSAo= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNwKW7o/izMnyH1Q74VDiFWfz/BCGC7DXFHyfUX0tcoJpSrui5C1nhVj4u1Ce7yT2gQTde7CdQrlh82g6pQxwOw= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-09_19:44:16 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-10T07:21:28.119 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa; will attempt to use it 2024-02-10T07:21:28.120 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa/tasks 2024-02-10T07:21:28.120 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-10T07:21:28.122 INFO:teuthology.task.internal:Checking packages... 2024-02-10T07:21:28.149 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90' 2024-02-10T07:21:28.149 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-10T07:21:28.149 INFO:teuthology.packaging:ref: None 2024-02-10T07:21:28.149 INFO:teuthology.packaging:tag: None 2024-02-10T07:21:28.149 INFO:teuthology.packaging:branch: main 2024-02-10T07:21:28.149 INFO:teuthology.packaging:sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:21:28.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-02-10T07:21:28.454 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1361.gf8e4e331 2024-02-10T07:21:28.456 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-10T07:21:28.462 INFO:teuthology.task.internal:no buildpackages task found 2024-02-10T07:21:28.462 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-10T07:21:28.469 INFO:teuthology.task.internal:Saving configuration 2024-02-10T07:21:28.480 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-10T07:21:28.486 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T07:21:28.512 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:16:26.337229', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6t5GCr8qlR/nF+e2IdSDCRLhpYRjYi9g1KBMBI9nehO22Aod6MfVunPPjNGq/ORhO4eH+cNSTIvWQJei7iSAo='} 2024-02-10T07:21:28.539 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:16:26.339367', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNwKW7o/izMnyH1Q74VDiFWfz/BCGC7DXFHyfUX0tcoJpSrui5C1nhVj4u1Ce7yT2gQTde7CdQrlh82g6pQxwOw='} 2024-02-10T07:21:28.539 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-10T07:21:28.546 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-10T07:21:28.546 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-10T07:21:28.546 INFO:teuthology.run_tasks:Running task console_log... 2024-02-10T07:21:28.623 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdeebb304c0>, signals=[15]) 2024-02-10T07:21:28.624 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-10T07:21:28.629 INFO:teuthology.task.internal:Opening connections... 2024-02-10T07:21:28.630 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-02-10T07:21:28.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:21:28.723 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-02-10T07:21:28.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:21:28.814 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-10T07:21:28.821 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-02-10T07:21:28.838 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-02-10T07:21:28.839 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-02-10T07:21:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-02-10T07:21:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-02-10T07:21:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-02-10T07:21:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-02-10T07:21:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-02-10T07:21:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-02-10T07:21:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T07:21:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-02-10T07:21:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-02-10T07:21:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T07:21:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-02-10T07:21:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T07:21:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T07:21:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T07:21:28.898 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-02-10T07:21:28.922 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-02-10T07:21:28.939 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-02-10T07:21:28.939 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-02-10T07:21:28.996 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-02-10T07:21:28.996 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T07:21:28.997 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T07:21:28.998 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-02-10T07:21:29.021 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-10T07:21:29.031 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-10T07:21:29.037 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-10T07:21:29.037 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-02-10T07:21:29.040 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-02-10T07:21:29.056 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-10T07:21:29.062 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-10T07:21:29.062 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-02-10T07:21:29.097 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-02-10T07:21:29.189 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-10T07:21:29.245 INFO:teuthology.run_tasks:Running task kernel... 2024-02-10T07:21:29.261 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-10T07:21:29.261 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-10T07:21:29.261 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T07:21:29.262 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T07:21:29.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:21:29.277 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-02-10T07:21:29.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:21:29.278 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-02-10T07:21:29.333 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-361.el9.x86_64 2024-02-10T07:21:29.333 INFO:teuthology.task.kernel:Running kernel on smithi155: 5.14.0-361.el9.x86_64 2024-02-10T07:21:29.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum install -y kernel 2024-02-10T07:21:29.336 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-361.el9.x86_64 2024-02-10T07:21:29.336 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-361.el9.x86_64 2024-02-10T07:21:29.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-02-10T07:21:31.481 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-10T07:21:31.518 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-10T07:21:32.890 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-02-10T07:21:32.893 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-02-10T07:21:34.474 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-02-10T07:21:35.134 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-02-10T07:21:37.642 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-02-10T07:21:41.640 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.9 MB 00:00 2024-02-10T07:21:42.633 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 19 MB 00:06 2024-02-10T07:21:43.266 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-02-10T07:21:47.728 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 20 MB 00:04 2024-02-10T07:21:47.883 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 5.9 MB 00:01 2024-02-10T07:21:49.418 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-02-10T07:21:52.658 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-10T07:21:54.907 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T07:21:54.907 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T07:21:55.023 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:21:55.027 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:21:55.027 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:21:55.027 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:21:55.027 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout:Install 4 Packages 2024-02-10T07:21:55.028 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:21:55.029 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 99 M 2024-02-10T07:21:55.029 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 122 M 2024-02-10T07:21:55.029 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:21:55.743 INFO:teuthology.orchestra.run.smithi155.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 8.4 MB/s | 6.0 MB 00:00 2024-02-10T07:21:56.651 INFO:teuthology.orchestra.run.smithi155.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 13 MB/s | 21 MB 00:01 2024-02-10T07:21:56.787 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 20 MB 00:07 2024-02-10T07:21:57.493 INFO:teuthology.orchestra.run.smithi155.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-02-10T07:21:57.793 INFO:teuthology.orchestra.run.smithi155.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 16 MB/s | 33 MB 00:02 2024-02-10T07:21:57.796 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:21:57.796 INFO:teuthology.orchestra.run.smithi155.stdout:Total 36 MB/s | 99 MB 00:02 2024-02-10T07:21:57.797 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:21:58.188 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:21:58.188 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:21:59.239 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:21:59.239 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:22:01.640 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:22:01.642 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-02-10T07:22:02.314 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:22:02.451 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:22:03.907 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T07:22:03.907 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T07:22:03.997 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:22:04.002 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:22:04.002 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:22:04.002 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:22:04.002 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:22:04.002 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-02-10T07:22:04.003 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:22:04.004 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 99 M 2024-02-10T07:22:04.004 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 122 M 2024-02-10T07:22:04.004 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:22:04.418 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 15 MB/s | 6.0 MB 00:00 2024-02-10T07:22:04.868 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 24 MB/s | 21 MB 00:00 2024-02-10T07:22:05.225 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:22:05.427 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 27 MB/s | 39 MB 00:01 2024-02-10T07:22:05.520 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:22:05.894 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 23 MB/s | 33 MB 00:01 2024-02-10T07:22:05.897 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:22:05.897 INFO:teuthology.orchestra.run.smithi070.stdout:Total 52 MB/s | 99 MB 00:01 2024-02-10T07:22:05.897 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:22:06.274 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:22:06.274 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:22:07.287 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:22:07.287 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:22:09.666 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:22:10.236 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:22:10.310 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:22:10.444 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:22:10.507 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:13.192 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:22:13.362 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:22:15.247 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:18.072 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:22:18.250 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:22.975 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:44.435 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:44.436 INFO:teuthology.orchestra.run.smithi155.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T07:22:44.436 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:22:44.506 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:44.912 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:44.912 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:22:44.912 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:22:44.912 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:22:45.360 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:45.360 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:22:45.360 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:22:45.360 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T07:22:45.360 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T07:22:45.361 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T07:22:45.361 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T07:22:45.361 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:22:45.361 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:22:45.896 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:22:46.058 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:22:46.059 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:22:46.059 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T07:22:46.059 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T07:22:46.059 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T07:22:46.060 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-10T07:22:46.060 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T07:22:46.060 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:22:46.168 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:22:46.169 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:22:46.169 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -qi grub2-tools 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Name : grub2-tools 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Epoch : 1 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Version : 2.06 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Release : 68.el9 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: x86_64 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Group : Unspecified 2024-02-10T07:22:46.210 INFO:teuthology.orchestra.run.smithi155.stdout:Size : 8264254 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:License : GPLv3+ 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Packager : builder@centos.org 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Vendor : CentOS 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Summary : Support tools for GRUB. 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:Description : 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout:hardware devices. 2024-02-10T07:22:46.211 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:22:46.212 INFO:teuthology.orchestra.run.smithi155.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T07:22:46.212 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T07:22:46.213 INFO:teuthology.task.kernel:Updating grub on smithi155 to boot 5.14.0-419.el9.x86_64 2024-02-10T07:22:46.213 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T07:22:46.848 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-02-10T07:22:48.243 INFO:teuthology.orchestra.run.smithi155.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T07:22:48.272 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-02-10T07:22:48.275 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-02-10T07:22:48.289 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.fYpCWoZQB7 2024-02-10T07:22:48.290 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fYpCWoZQB7 2024-02-10T07:22:48.359 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.fYpCWoZQB7 2024-02-10T07:22:48.500 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.fYpCWoZQB7 is 6KB 2024-02-10T07:22:48.548 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.fYpCWoZQB7 2024-02-10T07:22:48.563 DEBUG:teuthology.orchestra.run.smithi155:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T07:22:48.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T07:22:48.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T07:22:48.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T07:22:48.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T07:22:48.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T07:22:48.790 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-02-10T07:22:51.917 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:51.917 INFO:teuthology.orchestra.run.smithi070.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T07:22:51.917 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:22:51.987 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:52.464 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:22:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:22:52.501 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:22:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:22:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:22:52.920 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:22:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T07:22:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T07:22:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T07:22:52.920 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T07:22:52.921 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:22:52.921 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:22:53.415 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:22:53.544 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:22:53.544 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:22:53.544 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T07:22:53.544 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T07:22:53.545 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T07:22:53.545 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-10T07:22:53.545 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T07:22:53.545 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:22:53.653 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:22:53.653 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-02-10T07:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-02-10T07:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-02-10T07:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-02-10T07:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 68.el9 2024-02-10T07:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-02-10T07:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8264254 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T07:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T07:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T07:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-02-10T07:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T07:22:53.700 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T07:22:53.700 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-419.el9.x86_64 2024-02-10T07:22:53.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T07:22:54.273 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-02-10T07:22:55.610 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T07:22:55.643 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-02-10T07:22:55.645 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-02-10T07:22:55.661 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.gAp50kOPGr 2024-02-10T07:22:55.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gAp50kOPGr 2024-02-10T07:22:55.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.gAp50kOPGr 2024-02-10T07:22:55.871 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.gAp50kOPGr is 6KB 2024-02-10T07:22:55.890 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.gAp50kOPGr 2024-02-10T07:22:55.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T07:22:55.972 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T07:22:55.973 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T07:22:55.973 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T07:22:55.973 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T07:22:55.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T07:22:56.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-02-10T07:23:18.816 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-10T07:23:18.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-10T07:23:18.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:23:26.132 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-10T07:23:26.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-02-10T07:23:26.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:23:53.277 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-02-10T07:24:02.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-10T07:24:02.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:24:20.668 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-02-10T07:24:26.140 DEBUG:teuthology.orchestra.remote:timed out 2024-02-10T07:24:32.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-10T07:24:32.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:24:33.309 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-02-10T07:24:33.711 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-10T07:24:33.711 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-419.el9.x86_64"... 2024-02-10T07:24:33.712 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-02-10T07:24:33.726 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-419.el9.x86_64 2024-02-10T07:24:33.726 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T07:24:33.726 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T07:24:33.726 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T07:24:34.728 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-10T07:24:34.728 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T07:24:34.816 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-02-10T07:24:34.851 DEBUG:teuthology.parallel:result is None 2024-02-10T07:24:35.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-02-10T07:24:35.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:24:35.540 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-02-10T07:24:35.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-02-10T07:24:35.981 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-419.el9.x86_64"... 2024-02-10T07:24:35.981 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-02-10T07:24:35.995 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-419.el9.x86_64 2024-02-10T07:24:35.995 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T07:24:35.996 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T07:24:35.996 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T07:24:36.997 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-10T07:24:36.997 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T07:24:37.066 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-02-10T07:24:37.105 DEBUG:teuthology.parallel:result is None 2024-02-10T07:24:37.105 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-10T07:24:37.116 INFO:teuthology.task.internal:Creating test directory... 2024-02-10T07:24:37.116 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T07:24:37.118 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T07:24:37.133 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-10T07:24:37.139 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-10T07:24:37.151 INFO:teuthology.task.internal:Creating archive directory... 2024-02-10T07:24:37.152 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T07:24:37.173 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T07:24:37.200 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-10T07:24:37.207 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-10T07:24:37.207 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T07:24:37.238 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T07:24:37.268 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:24:37.269 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:24:37.281 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:24:37.281 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:24:37.283 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-10T07:24:37.290 INFO:teuthology.task.internal:Configuring sudo... 2024-02-10T07:24:37.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T07:24:37.326 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T07:24:37.356 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-10T07:24:37.365 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-10T07:24:37.365 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T07:24:37.393 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T07:24:37.414 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:24:37.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:24:37.528 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:24:37.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:24:37.646 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:24:37.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T07:24:37.710 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:24:37.752 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:24:37.803 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:24:37.887 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:24:37.932 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:24:37.933 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T07:24:38.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-02-10T07:24:38.006 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-02-10T07:24:38.062 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:24:38.095 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:24:38.588 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-10T07:24:38.597 INFO:teuthology.task.internal:Starting timer... 2024-02-10T07:24:38.597 INFO:teuthology.run_tasks:Running task pcp... 2024-02-10T07:24:38.613 INFO:teuthology.run_tasks:Running task selinux... 2024-02-10T07:24:38.627 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-10T07:24:38.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-02-10T07:24:38.690 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-02-10T07:24:38.691 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-02-10T07:24:38.840 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-02-10T07:24:38.842 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-10T07:24:38.842 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-02-10T07:24:38.862 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-02-10T07:24:38.862 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-02-10T07:24:38.910 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-02-10T07:24:38.910 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-02-10T07:24:38.910 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:24:38.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:24:38.937 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:24:38.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:24:38.963 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-10T07:24:38.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-02-10T07:24:39.056 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-02-10T07:24:39.269 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-10T07:24:39.280 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-10T07:24:39.285 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-10T07:24:39.419 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-10T07:24:39.435 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-10T07:24:39.436 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-10T07:28:54.130 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-02-10T07:28:54.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-02-10T07:28:54.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:28:54.223 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-02-10T07:28:54.302 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-02-10T07:28:54.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-10T07:28:54.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:28:54.391 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-02-10T07:28:54.470 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-10T07:28:54.471 INFO:teuthology.run_tasks:Running task clock... 2024-02-10T07:28:54.482 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-10T07:28:54.482 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T07:28:54.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:28:54.485 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T07:28:54.485 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:28:54.513 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T07:28:54.530 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T07:28:54.542 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T07:28:54.558 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T07:28:54.559 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-02-10T07:28:54.572 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-02-10T07:28:54.587 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-02-10T07:28:54.588 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T07:28:54.601 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-02-10T07:28:54.604 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T07:28:54.617 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T07:28:54.634 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T07:28:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:28:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:28:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-02-10T07:28:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.658 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.659 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.685 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:28:54.688 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:28:54.688 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-02-10T07:28:54.688 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.688 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.689 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.689 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:28:54.689 INFO:teuthology.run_tasks:Running task install... 2024-02-10T07:28:54.697 DEBUG:teuthology.task.install:project ceph 2024-02-10T07:28:54.698 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'}} 2024-02-10T07:28:54.698 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'} 2024-02-10T07:28:54.698 INFO:teuthology.task.install:Using flavor: default 2024-02-10T07:28:54.704 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-10T07:28:54.704 INFO:teuthology.task.install:extra packages: [] 2024-02-10T07:28:54.704 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'tag': None, 'wait_for_package': False} 2024-02-10T07:28:54.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:28:54.707 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'tag': None, 'wait_for_package': False} 2024-02-10T07:28:54.707 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:28:54.946 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/ 2024-02-10T07:28:54.947 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1361.g290212a1 2024-02-10T07:28:54.953 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/ 2024-02-10T07:28:54.953 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1361.g290212a1 2024-02-10T07:28:55.144 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-10T07:28:55.145 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:28:55.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-10T07:28:55.148 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-10T07:28:55.148 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:28:55.148 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-10T07:28:55.176 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-10T07:28:55.177 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-10T07:28:55.177 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-10T07:28:55.178 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-10T07:28:55.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-10T07:28:55.246 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-10T07:28:55.325 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-10T07:28:55.326 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-10T07:28:55.391 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-02-10T07:28:55.391 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-02-10T07:28:55.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-02-10T07:28:55.394 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-02-10T07:28:55.632 INFO:teuthology.orchestra.run.smithi070.stdout:74 files removed 2024-02-10T07:28:55.648 INFO:teuthology.orchestra.run.smithi155.stdout:74 files removed 2024-02-10T07:28:55.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw 2024-02-10T07:28:55.674 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw 2024-02-10T07:28:56.564 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 139 kB/s | 82 kB 00:00 2024-02-10T07:28:56.578 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 140 kB/s | 82 kB 00:00 2024-02-10T07:28:57.014 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-02-10T07:28:57.029 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-02-10T07:28:57.373 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-02-10T07:28:57.379 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-02-10T07:28:57.849 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-02-10T07:28:57.850 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-10T07:28:59.259 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-02-10T07:28:59.366 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-02-10T07:29:01.441 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-02-10T07:29:02.023 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-02-10T07:29:03.175 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-02-10T07:29:07.118 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.9 MB 00:00 2024-02-10T07:29:08.502 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 15 kB 00:00 2024-02-10T07:29:10.461 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-10T07:29:15.394 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-02-10T07:29:18.055 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:29:18.063 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:29:18.063 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:29:18.064 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:29:18.064 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:29:18.064 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 9.7 M 2024-02-10T07:29:18.064 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-02-10T07:29:18.064 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.4 M 2024-02-10T07:29:18.065 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T07:29:18.065 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:29:18.065 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options 2024-02-10T07:29:18.065 INFO:teuthology.orchestra.run.smithi070.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-10T07:29:18.065 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.3 M 2024-02-10T07:29:18.065 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 20 M 2024-02-10T07:29:18.065 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.0.0-1361.g290212a1.el9 ceph 25 k 2024-02-10T07:29:18.066 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-10T07:29:18.066 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-10T07:29:18.066 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-10T07:29:18.066 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 743 k 2024-02-10T07:29:18.066 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-10T07:29:18.066 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 499 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.1 M 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: x86_64 2:19.0.0-1361.g290212a1.el9 ceph 45 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 131 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.0.0-1361.g290212a1.el9 ceph 161 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.0.0-1361.g290212a1.el9 ceph 319 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 298 k 2024-02-10T07:29:18.067 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 99 k 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout:Install 30 Packages 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-02-10T07:29:18.068 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:18.069 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 58 M 2024-02-10T07:29:18.069 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:29:19.239 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-02-10T07:29:19.280 INFO:teuthology.orchestra.run.smithi070.stdout:(1/32): ceph-base-19.0.0-1361.g290212a1.el9.x86 5.2 MB/s | 5.3 MB 00:01 2024-02-10T07:29:19.347 INFO:teuthology.orchestra.run.smithi070.stdout:(2/32): ceph-selinux-19.0.0-1361.g290212a1.el9. 379 kB/s | 25 kB 00:00 2024-02-10T07:29:19.481 INFO:teuthology.orchestra.run.smithi070.stdout:(3/32): libcephfs2-19.0.0-1361.g290212a1.el9.x8 5.4 MB/s | 743 kB 00:00 2024-02-10T07:29:19.590 INFO:teuthology.orchestra.run.smithi070.stdout:(4/32): ceph-radosgw-19.0.0-1361.g290212a1.el9. 7.3 MB/s | 9.7 MB 00:01 2024-02-10T07:29:19.623 INFO:teuthology.orchestra.run.smithi070.stdout:(5/32): libradosstriper1-19.0.0-1361.g290212a1. 3.4 MB/s | 499 kB 00:00 2024-02-10T07:29:19.706 INFO:teuthology.orchestra.run.smithi070.stdout:(6/32): python3-ceph-argparse-19.0.0-1361.g2902 542 kB/s | 45 kB 00:00 2024-02-10T07:29:19.798 INFO:teuthology.orchestra.run.smithi070.stdout:(7/32): python3-ceph-common-19.0.0-1361.g290212 1.4 MB/s | 131 kB 00:00 2024-02-10T07:29:19.882 INFO:teuthology.orchestra.run.smithi070.stdout:(8/32): python3-cephfs-19.0.0-1361.g290212a1.el 1.9 MB/s | 161 kB 00:00 2024-02-10T07:29:19.974 INFO:teuthology.orchestra.run.smithi070.stdout:(9/32): python3-rados-19.0.0-1361.g290212a1.el9 3.4 MB/s | 319 kB 00:00 2024-02-10T07:29:20.041 INFO:teuthology.orchestra.run.smithi070.stdout:(10/32): librgw2-19.0.0-1361.g290212a1.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-02-10T07:29:20.066 INFO:teuthology.orchestra.run.smithi070.stdout:(11/32): python3-rbd-19.0.0-1361.g290212a1.el9. 3.2 MB/s | 298 kB 00:00 2024-02-10T07:29:20.100 INFO:teuthology.orchestra.run.smithi070.stdout:(12/32): python3-rgw-19.0.0-1361.g290212a1.el9. 1.7 MB/s | 99 kB 00:00 2024-02-10T07:29:20.367 INFO:teuthology.orchestra.run.smithi070.stdout:(13/32): ceph-common-19.0.0-1361.g290212a1.el9. 9.4 MB/s | 20 MB 00:02 2024-02-10T07:29:20.425 INFO:teuthology.orchestra.run.smithi070.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 295 kB/s | 106 kB 00:00 2024-02-10T07:29:20.484 INFO:teuthology.orchestra.run.smithi070.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 383 kB/s | 44 kB 00:00 2024-02-10T07:29:20.509 INFO:teuthology.orchestra.run.smithi070.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 413 kB/s | 159 kB 00:00 2024-02-10T07:29:20.909 INFO:teuthology.orchestra.run.smithi070.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 661 kB 00:00 2024-02-10T07:29:20.951 INFO:teuthology.orchestra.run.smithi070.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 605 kB/s | 282 kB 00:00 2024-02-10T07:29:20.977 INFO:teuthology.orchestra.run.smithi070.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 88 kB/s | 41 kB 00:00 2024-02-10T07:29:21.002 INFO:teuthology.orchestra.run.smithi070.stdout:(20/32): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 215 kB 00:00 2024-02-10T07:29:21.027 INFO:teuthology.orchestra.run.smithi070.stdout:(21/32): qatlib-service-23.11.0-1.el9.x86_64.rp 449 kB/s | 34 kB 00:00 2024-02-10T07:29:21.052 INFO:teuthology.orchestra.run.smithi070.stdout:(22/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 580 kB/s | 43 kB 00:00 2024-02-10T07:29:21.319 INFO:teuthology.orchestra.run.smithi070.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-02-10T07:29:21.502 INFO:teuthology.orchestra.run.smithi070.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 265 kB/s | 49 kB 00:00 2024-02-10T07:29:21.561 INFO:teuthology.orchestra.run.smithi070.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 579 kB/s | 309 kB 00:00 2024-02-10T07:29:21.695 INFO:teuthology.orchestra.run.smithi070.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 351 kB/s | 67 kB 00:00 2024-02-10T07:29:21.828 INFO:teuthology.orchestra.run.smithi070.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-02-10T07:29:21.878 INFO:teuthology.orchestra.run.smithi070.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2024-02-10T07:29:21.945 INFO:teuthology.orchestra.run.smithi070.stdout:(29/32): libarrow-9.0.0-10.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-02-10T07:29:22.029 INFO:teuthology.orchestra.run.smithi070.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-02-10T07:29:22.279 INFO:teuthology.orchestra.run.smithi070.stdout:(31/32): librbd1-19.0.0-1361.g290212a1.el9.x86_ 9.4 MB/s | 3.1 MB 00:00 2024-02-10T07:29:22.396 INFO:teuthology.orchestra.run.smithi070.stdout:(32/32): librados2-19.0.0-1361.g290212a1.el9.x8 6.5 MB/s | 3.4 MB 00:00 2024-02-10T07:29:22.400 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:29:22.400 INFO:teuthology.orchestra.run.smithi070.stdout:Total 13 MB/s | 58 MB 00:04 2024-02-10T07:29:22.401 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:29:22.438 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:29:22.438 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:29:22.911 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:29:22.912 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:29:23.156 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 15 MB/s | 5.9 MB 00:00 2024-02-10T07:29:24.378 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:29:24.458 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-10T07:29:24.492 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-10T07:29:24.518 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-10T07:29:24.540 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T07:29:24.557 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T07:29:24.565 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 15 kB 00:00 2024-02-10T07:29:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T07:29:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T07:29:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-10T07:29:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T07:29:25.294 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T07:29:25.375 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 8/34 2024-02-10T07:29:25.404 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T07:29:25.458 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T07:29:25.495 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-10T07:29:25.539 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 11/34 2024-02-10T07:29:25.608 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T07:29:25.631 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T07:29:25.693 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T07:29:25.720 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-10T07:29:25.828 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-10T07:29:25.863 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-10T07:29:25.903 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-10T07:29:25.928 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-10T07:29:25.957 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-10T07:29:25.990 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-10T07:29:26.308 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-10T07:29:26.336 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T07:29:26.410 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T07:29:26.441 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 23/34 2024-02-10T07:29:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-10T07:29:26.947 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-10T07:29:26.972 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T07:29:27.044 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T07:29:27.112 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 27/34 2024-02-10T07:29:27.141 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 28/34 2024-02-10T07:29:29.283 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T07:29:29.320 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T07:29:29.878 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T07:29:29.924 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T07:29:30.023 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T07:29:30.023 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-10T07:29:30.023 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-10T07:29:30.023 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:30.058 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T07:29:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-02-10T07:29:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:40.823 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T07:29:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T07:29:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:29:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-10T07:29:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T07:29:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T07:29:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T07:29:40.934 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T07:29:40.934 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T07:29:42.062 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T07:29:42.062 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 1/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 3/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 4/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 5/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 6/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 8/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 9/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 10/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 11/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 13/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-10T07:29:42.063 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-10T07:29:42.064 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-10T07:29:42.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-10T07:29:42.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-10T07:29:42.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-10T07:29:42.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T07:29:42.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-10T07:29:42.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 33/34 2024-02-10T07:29:43.232 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T07:29:43.232 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:43.232 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-02-10T07:29:43.232 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.232 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-10T07:29:43.233 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-10T07:29:43.234 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-3.el9.x86_64 2024-02-10T07:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-10T07:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:43.235 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:29:43.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-test 2024-02-10T07:29:44.179 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:29 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:29:44.336 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:29:44.337 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:29:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:29:44.337 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:29:44.337 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:29:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.0.0-1361.g290212a1.el9 ceph 47 M 2024-02-10T07:29:44.337 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.0.0-1361.g290212a1.el9 ceph 163 k 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-02-10T07:29:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:44.339 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 48 M 2024-02-10T07:29:44.339 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 204 M 2024-02-10T07:29:44.339 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:29:44.511 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 300 kB 00:00 2024-02-10T07:29:44.545 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-02-10T07:29:44.587 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-10T07:29:44.629 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-02-10T07:29:44.687 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): libcephsqlite-19.0.0-1361.g290212a1.el9. 470 kB/s | 163 kB 00:00 2024-02-10T07:29:45.236 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 20 MB 00:20 2024-02-10T07:29:46.796 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): ceph-test-19.0.0-1361.g290212a1.el9.x86_ 19 MB/s | 47 MB 00:02 2024-02-10T07:29:46.799 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:29:46.799 INFO:teuthology.orchestra.run.smithi070.stdout:Total 20 MB/s | 48 MB 00:02 2024-02-10T07:29:46.799 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:29:46.819 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:29:46.819 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:29:47.219 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:29:47.219 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:29:47.823 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:29:47.877 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-10T07:29:47.914 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-10T07:29:47.956 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-10T07:29:48.006 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-10T07:29:48.032 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T07:29:50.225 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-10T07:29:52.522 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T07:29:52.597 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T07:29:52.920 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:29:52.929 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:29:52.929 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:29:52.929 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:29:52.929 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:29:52.929 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 9.7 M 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.4 M 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.3 M 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 20 M 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.0.0-1361.g290212a1.el9 ceph 25 k 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-10T07:29:52.930 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 743 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 499 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.0.0-1361.g290212a1.el9 ceph 5.1 M 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse 2024-02-10T07:29:52.931 INFO:teuthology.orchestra.run.smithi155.stdout: x86_64 2:19.0.0-1361.g290212a1.el9 ceph 45 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.0.0-1361.g290212a1.el9 ceph 131 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.0.0-1361.g290212a1.el9 ceph 161 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.0.0-1361.g290212a1.el9 ceph 319 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 298 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.0.0-1361.g290212a1.el9 ceph 99 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-02-10T07:29:52.932 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout:Install 30 Packages 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 2 Packages 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 58 M 2024-02-10T07:29:52.933 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:29:53.817 INFO:teuthology.orchestra.run.smithi155.stdout:(1/32): ceph-base-19.0.0-1361.g290212a1.el9.x86 6.4 MB/s | 5.3 MB 00:00 2024-02-10T07:29:53.893 INFO:teuthology.orchestra.run.smithi155.stdout:(2/32): ceph-selinux-19.0.0-1361.g290212a1.el9. 337 kB/s | 25 kB 00:00 2024-02-10T07:29:54.018 INFO:teuthology.orchestra.run.smithi155.stdout:(3/32): ceph-radosgw-19.0.0-1361.g290212a1.el9. 9.3 MB/s | 9.7 MB 00:01 2024-02-10T07:29:54.060 INFO:teuthology.orchestra.run.smithi155.stdout:(4/32): libcephfs2-19.0.0-1361.g290212a1.el9.x8 4.3 MB/s | 743 kB 00:00 2024-02-10T07:29:54.094 INFO:teuthology.orchestra.run.smithi155.stdout:(5/32): libradosstriper1-19.0.0-1361.g290212a1. 6.5 MB/s | 499 kB 00:00 2024-02-10T07:29:54.160 INFO:teuthology.orchestra.run.smithi155.stdout:(6/32): python3-ceph-argparse-19.0.0-1361.g2902 678 kB/s | 45 kB 00:00 2024-02-10T07:29:54.271 INFO:teuthology.orchestra.run.smithi155.stdout:(7/32): python3-ceph-common-19.0.0-1361.g290212 1.2 MB/s | 131 kB 00:00 2024-02-10T07:29:54.353 INFO:teuthology.orchestra.run.smithi155.stdout:(8/32): python3-cephfs-19.0.0-1361.g290212a1.el 1.9 MB/s | 161 kB 00:00 2024-02-10T07:29:54.428 INFO:teuthology.orchestra.run.smithi155.stdout:(9/32): python3-rados-19.0.0-1361.g290212a1.el9 4.2 MB/s | 319 kB 00:00 2024-02-10T07:29:54.495 INFO:teuthology.orchestra.run.smithi155.stdout:(10/32): librgw2-19.0.0-1361.g290212a1.el9.x86_ 12 MB/s | 5.1 MB 00:00 2024-02-10T07:29:54.520 INFO:teuthology.orchestra.run.smithi155.stdout:(11/32): python3-rbd-19.0.0-1361.g290212a1.el9. 3.2 MB/s | 298 kB 00:00 2024-02-10T07:29:54.554 INFO:teuthology.orchestra.run.smithi155.stdout:(12/32): python3-rgw-19.0.0-1361.g290212a1.el9. 1.7 MB/s | 99 kB 00:00 2024-02-10T07:29:54.671 INFO:teuthology.orchestra.run.smithi155.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 704 kB/s | 106 kB 00:00 2024-02-10T07:29:54.696 INFO:teuthology.orchestra.run.smithi155.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-02-10T07:29:54.713 INFO:teuthology.orchestra.run.smithi155.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-10T07:29:54.796 INFO:teuthology.orchestra.run.smithi155.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 661 kB 00:00 2024-02-10T07:29:54.822 INFO:teuthology.orchestra.run.smithi155.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 282 kB 00:00 2024-02-10T07:29:54.847 INFO:teuthology.orchestra.run.smithi155.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 823 kB/s | 41 kB 00:00 2024-02-10T07:29:54.889 INFO:teuthology.orchestra.run.smithi155.stdout:(19/32): qatlib-23.11.0-1.el9.x86_64.rpm 3.1 MB/s | 215 kB 00:00 2024-02-10T07:29:54.906 INFO:teuthology.orchestra.run.smithi155.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 574 kB/s | 34 kB 00:00 2024-02-10T07:29:54.940 INFO:teuthology.orchestra.run.smithi155.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 862 kB/s | 43 kB 00:00 2024-02-10T07:29:55.032 INFO:teuthology.orchestra.run.smithi155.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.4 MB/s | 309 kB 00:00 2024-02-10T07:29:55.054 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T07:29:55.055 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 1/6 2024-02-10T07:29:55.055 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2/6 2024-02-10T07:29:55.055 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-10T07:29:55.055 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-10T07:29:55.055 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-10T07:29:55.065 INFO:teuthology.orchestra.run.smithi155.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 783 kB/s | 26 kB 00:00 2024-02-10T07:29:55.099 INFO:teuthology.orchestra.run.smithi155.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-02-10T07:29:55.132 INFO:teuthology.orchestra.run.smithi155.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-02-10T07:29:55.208 INFO:teuthology.orchestra.run.smithi155.stdout:(26/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-02-10T07:29:55.250 INFO:teuthology.orchestra.run.smithi155.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-02-10T07:29:55.317 INFO:teuthology.orchestra.run.smithi155.stdout:(28/32): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-02-10T07:29:55.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: jq-1.6-16.el9.x86_64 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:55.475 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:29:55.509 INFO:teuthology.orchestra.run.smithi155.stdout:(29/32): ceph-common-19.0.0-1361.g290212a1.el9. 7.9 MB/s | 20 MB 00:02 2024-02-10T07:29:55.709 INFO:teuthology.orchestra.run.smithi155.stdout:(30/32): libarrow-9.0.0-10.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-02-10T07:29:55.767 INFO:teuthology.orchestra.run.smithi155.stdout:(31/32): librados2-19.0.0-1361.g290212a1.el9.x8 7.5 MB/s | 3.4 MB 00:00 2024-02-10T07:29:55.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph 2024-02-10T07:29:55.826 INFO:teuthology.orchestra.run.smithi155.stdout:(32/32): librbd1-19.0.0-1361.g290212a1.el9.x86_ 9.9 MB/s | 3.1 MB 00:00 2024-02-10T07:29:55.829 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:29:55.830 INFO:teuthology.orchestra.run.smithi155.stdout:Total 20 MB/s | 58 MB 00:02 2024-02-10T07:29:55.830 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:29:55.863 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:29:55.863 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:29:56.337 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:29:56.337 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:29:56.363 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:41 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:29:56.527 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:29:56.529 INFO:teuthology.orchestra.run.smithi070.stdout:========================================================================================= 2024-02-10T07:29:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:29:56.529 INFO:teuthology.orchestra.run.smithi070.stdout:========================================================================================= 2024-02-10T07:29:56.529 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:29:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.0.0-1361.g290212a1.el9 ceph 6.6 k 2024-02-10T07:29:56.529 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:29:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.0.0-1361.g290212a1.el9 ceph 2.1 M 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.0.0-1361.g290212a1.el9 ceph 1.5 M 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 247 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.0.0-1361.g290212a1.el9 ceph 4.7 M 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 17 M 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-10T07:29:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-10T07:29:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout:========================================================================================= 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout:Install 36 Packages 2024-02-10T07:29:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:29:56.533 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 30 M 2024-02-10T07:29:56.533 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 107 M 2024-02-10T07:29:56.533 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:29:57.055 INFO:teuthology.orchestra.run.smithi070.stdout:(1/36): ceph-19.0.0-1361.g290212a1.el9.x86_64.r 39 kB/s | 6.6 kB 00:00 2024-02-10T07:29:57.313 INFO:teuthology.orchestra.run.smithi070.stdout:(2/36): ceph-mds-19.0.0-1361.g290212a1.el9.x86_ 5.0 MB/s | 2.1 MB 00:00 2024-02-10T07:29:57.414 INFO:teuthology.orchestra.run.smithi070.stdout:(3/36): ceph-mgr-19.0.0-1361.g290212a1.el9.x86_ 2.8 MB/s | 1.5 MB 00:00 2024-02-10T07:29:57.497 INFO:teuthology.orchestra.run.smithi070.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1361.g2902 2.9 MB/s | 247 kB 00:00 2024-02-10T07:29:57.706 INFO:teuthology.orchestra.run.smithi070.stdout:(5/36): ceph-mon-19.0.0-1361.g290212a1.el9.x86_ 7.2 MB/s | 4.7 MB 00:00 2024-02-10T07:29:57.731 INFO:teuthology.orchestra.run.smithi070.stdout:(6/36): lua-5.4.4-4.el9.x86_64.rpm 799 kB/s | 187 kB 00:00 2024-02-10T07:29:57.764 INFO:teuthology.orchestra.run.smithi070.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-02-10T07:29:57.790 INFO:teuthology.orchestra.run.smithi070.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 541 kB/s | 32 kB 00:00 2024-02-10T07:29:57.815 INFO:teuthology.orchestra.run.smithi070.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 941 kB/s | 47 kB 00:00 2024-02-10T07:29:57.834 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:29:57.840 INFO:teuthology.orchestra.run.smithi070.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 769 kB/s | 39 kB 00:00 2024-02-10T07:29:57.866 INFO:teuthology.orchestra.run.smithi070.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-10T07:29:57.916 INFO:teuthology.orchestra.run.smithi070.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-02-10T07:29:57.941 INFO:teuthology.orchestra.run.smithi070.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-02-10T07:29:57.966 INFO:teuthology.orchestra.run.smithi070.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-02-10T07:29:57.992 INFO:teuthology.orchestra.run.smithi070.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-10T07:29:57.994 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-10T07:29:58.017 INFO:teuthology.orchestra.run.smithi070.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-10T07:29:58.098 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-10T07:29:58.115 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-10T07:29:58.137 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T07:29:58.151 INFO:teuthology.orchestra.run.smithi070.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-02-10T07:29:58.156 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-10T07:29:58.316 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T07:29:58.318 INFO:teuthology.orchestra.run.smithi070.stdout:(18/36): ceph-osd-19.0.0-1361.g290212a1.el9.x86 17 MB/s | 17 MB 00:01 2024-02-10T07:29:58.370 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-10T07:29:58.618 INFO:teuthology.orchestra.run.smithi070.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 93 kB/s | 43 kB 00:00 2024-02-10T07:29:58.651 INFO:teuthology.orchestra.run.smithi070.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 238 kB/s | 151 kB 00:00 2024-02-10T07:29:58.696 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-10T07:29:58.760 INFO:teuthology.orchestra.run.smithi070.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 99 kB/s | 11 kB 00:00 2024-02-10T07:29:58.766 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T07:29:58.866 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T07:29:58.868 INFO:teuthology.orchestra.run.smithi070.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-02-10T07:29:58.964 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 8/34 2024-02-10T07:29:58.977 INFO:teuthology.orchestra.run.smithi070.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 214 kB/s | 23 kB 00:00 2024-02-10T07:29:58.992 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T07:29:59.035 INFO:teuthology.orchestra.run.smithi070.stdout:(24/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 239 kB/s | 172 kB 00:00 2024-02-10T07:29:59.038 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 9/34 2024-02-10T07:29:59.067 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-10T07:29:59.077 INFO:teuthology.orchestra.run.smithi070.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-02-10T07:29:59.111 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 11/34 2024-02-10T07:29:59.136 INFO:teuthology.orchestra.run.smithi070.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-02-10T07:29:59.179 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T07:29:59.203 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T07:29:59.257 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 13/34 2024-02-10T07:29:59.270 INFO:teuthology.orchestra.run.smithi070.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 241 kB/s | 46 kB 00:00 2024-02-10T07:29:59.284 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-10T07:29:59.337 INFO:teuthology.orchestra.run.smithi070.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 393 kB/s | 79 kB 00:00 2024-02-10T07:29:59.407 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-10T07:29:59.453 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-10T07:29:59.453 INFO:teuthology.orchestra.run.smithi070.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 141 kB/s | 16 kB 00:00 2024-02-10T07:29:59.493 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-10T07:29:59.526 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-10T07:29:59.555 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-10T07:29:59.562 INFO:teuthology.orchestra.run.smithi070.stdout:(30/36): python3-cherrypy-18.6.1-2.el9.noarch.r 379 kB/s | 358 kB 00:00 2024-02-10T07:29:59.587 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-10T07:29:59.671 INFO:teuthology.orchestra.run.smithi070.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 414 kB/s | 90 kB 00:00 2024-02-10T07:29:59.696 INFO:teuthology.orchestra.run.smithi070.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 268 kB/s | 36 kB 00:00 2024-02-10T07:29:59.830 INFO:teuthology.orchestra.run.smithi070.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 486 kB/s | 272 kB 00:00 2024-02-10T07:29:59.906 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-10T07:29:59.934 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T07:29:59.938 INFO:teuthology.orchestra.run.smithi070.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-02-10T07:30:00.007 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 22/34 2024-02-10T07:30:00.038 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 23/34 2024-02-10T07:30:00.081 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-10T07:30:00.147 INFO:teuthology.orchestra.run.smithi070.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 483 kB/s | 230 kB 00:00 2024-02-10T07:30:00.439 INFO:teuthology.orchestra.run.smithi070.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 576 kB/s | 427 kB 00:00 2024-02-10T07:30:00.443 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:00.443 INFO:teuthology.orchestra.run.smithi070.stdout:Total 7.6 MB/s | 30 MB 00:03 2024-02-10T07:30:00.459 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:00.502 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:00.502 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:00.537 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-10T07:30:00.561 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T07:30:00.632 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 26/34 2024-02-10T07:30:00.701 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 27/34 2024-02-10T07:30:00.730 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 28/34 2024-02-10T07:30:00.939 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:00.939 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:02.349 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-10T07:30:02.382 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-10T07:30:02.410 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-10T07:30:02.480 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-10T07:30:02.525 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-10T07:30:02.558 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-10T07:30:02.649 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-10T07:30:02.719 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-10T07:30:02.760 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-10T07:30:02.832 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-10T07:30:02.888 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-10T07:30:02.894 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T07:30:02.926 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-10T07:30:02.935 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T07:30:02.976 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-10T07:30:03.039 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-10T07:30:03.222 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-10T07:30:03.264 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-10T07:30:03.312 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-10T07:30:03.345 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-10T07:30:03.375 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-10T07:30:03.444 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-10T07:30:03.479 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 29/34 2024-02-10T07:30:03.494 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T07:30:03.513 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T07:30:03.528 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-10T07:30:03.568 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-10T07:30:03.603 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-10T07:30:03.610 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 30/34 2024-02-10T07:30:03.610 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-10T07:30:03.611 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-10T07:30:03.611 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:03.637 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-10T07:30:03.656 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T07:30:03.727 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-10T07:30:03.769 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-10T07:30:03.840 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-10T07:30:03.978 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-10T07:30:04.104 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T07:30:04.262 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 31/36 2024-02-10T07:30:04.290 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T07:30:04.326 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T07:30:04.326 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:04.326 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-10T07:30:04.326 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T07:30:04.326 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T07:30:04.326 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:05.774 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T07:30:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T07:30:05.801 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:05.801 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-10T07:30:05.802 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T07:30:05.802 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T07:30:05.802 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:06.241 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T07:30:06.264 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T07:30:06.264 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:06.264 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-10T07:30:06.264 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T07:30:06.265 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T07:30:06.265 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T07:30:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T07:30:06.490 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:06.490 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-10T07:30:06.490 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T07:30:06.490 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T07:30:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:06.557 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T07:30:08.414 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T07:30:08.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 1/36 2024-02-10T07:30:08.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 3/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 4/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 5/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 6/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-10T07:30:08.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-10T07:30:08.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-10T07:30:08.417 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-10T07:30:08.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-10T07:30:08.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T07:30:08.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-10T07:30:08.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-10T07:30:08.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-10T07:30:08.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-10T07:30:08.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-10T07:30:09.383 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-10T07:30:09.383 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: lua-5.4.4-4.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-10T07:30:09.384 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-10T07:30:09.385 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:09.386 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:09.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-base 2024-02-10T07:30:10.330 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:55 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:10.435 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:10.488 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:10.489 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:10.489 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:10.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install cephadm 2024-02-10T07:30:11.137 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:56 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:11.295 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:11.296 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 748 k 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 748 k 2024-02-10T07:30:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 754 k 2024-02-10T07:30:11.298 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:11.624 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm-19.0.0-1361.g290212a1.el9.noarch.rpm 2.2 MB/s | 748 kB 00:00 2024-02-10T07:30:11.625 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:11.625 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.2 MB/s | 748 kB 00:00 2024-02-10T07:30:11.625 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:11.630 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:11.630 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:11.635 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:11.636 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:11.708 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:12.065 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:12.543 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:12.863 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:12.864 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:12.864 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:12.864 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:12.864 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:12.864 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:13.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-immutable-object-cache 2024-02-10T07:30:13.392 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T07:30:13.392 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-02-10T07:30:13.392 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-02-10T07:30:13.392 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-02-10T07:30:13.392 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-02-10T07:30:13.393 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-02-10T07:30:13.393 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-02-10T07:30:13.393 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-02-10T07:30:13.393 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:13.691 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:58 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:13.847 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:13.847 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1361.g290212a1.el9 ceph 145 k 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 145 k 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 436 k 2024-02-10T07:30:13.848 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:14.092 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-immutable-object-cache-19.0.0-1361.g290212 598 kB/s | 145 kB 00:00 2024-02-10T07:30:14.093 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:14.093 INFO:teuthology.orchestra.run.smithi070.stdout:Total 594 kB/s | 145 kB 00:00 2024-02-10T07:30:14.093 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:14.100 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:14.100 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:14.151 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:14.152 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:14.288 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:14.326 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T07:30:14.334 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T07:30:14.350 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 32/34 2024-02-10T07:30:14.350 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:14.350 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-10T07:30:14.350 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T07:30:14.350 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-10T07:30:14.350 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:14.352 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T07:30:14.359 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T07:30:14.359 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:14.359 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-10T07:30:14.359 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:14.428 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-10T07:30:14.428 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T07:30:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T07:30:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:15.361 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:15.361 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:15.574 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T07:30:15.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 1/34 2024-02-10T07:30:15.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2/34 2024-02-10T07:30:15.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 3/34 2024-02-10T07:30:15.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 4/34 2024-02-10T07:30:15.574 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 5/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_6 6/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 7/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9. 8/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x8 9/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 10/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 11/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 12/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 13/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-10T07:30:15.575 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-10T07:30:15.576 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-10T07:30:15.577 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.0.0-1361.g290212a1.el9.x86_64 31/34 2024-02-10T07:30:15.577 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-10T07:30:15.577 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 33/34 2024-02-10T07:30:15.609 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr 2024-02-10T07:30:16.203 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:01 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:16.320 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:16.372 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:16.373 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:16.373 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:16.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-dashboard 2024-02-10T07:30:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-10T07:30:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:16.661 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-02-10T07:30:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.661 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.661 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-10T07:30:16.662 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.663 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.664 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-3.el9.x86_64 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:16.665 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:16.915 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-test 2024-02-10T07:30:17.029 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:02 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:17.188 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:17.189 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 1.8 M 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 28 k 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 15 k 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-10T07:30:17.190 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-02-10T07:30:17.191 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:17.192 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 6.8 M 2024-02-10T07:30:17.192 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 26 M 2024-02-10T07:30:17.192 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:17.498 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:27 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:17.554 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): ceph-grafana-dashboards-19.0.0-1361.g29 144 kB/s | 28 kB 00:00 2024-02-10T07:30:17.579 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): ceph-prometheus-alerts-19.0.0-1361.g290 69 kB/s | 15 kB 00:00 2024-02-10T07:30:17.655 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.0.0-1361.g290212a1.el9 ceph 47 M 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.0.0-1361.g290212a1.el9 ceph 163 k 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-10T07:30:17.656 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-10T07:30:17.657 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-10T07:30:17.657 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:17.657 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:30:17.657 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:17.657 INFO:teuthology.orchestra.run.smithi155.stdout:Install 6 Packages 2024-02-10T07:30:17.657 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:17.657 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 48 M 2024-02-10T07:30:17.658 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 204 M 2024-02-10T07:30:17.658 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:30:17.829 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.0 MB/s | 238 kB 00:00 2024-02-10T07:30:17.871 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): ceph-mgr-dashboard-19.0.0-1361.g290212a 3.4 MB/s | 1.8 MB 00:00 2024-02-10T07:30:17.889 INFO:teuthology.orchestra.run.smithi155.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-02-10T07:30:17.913 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-02-10T07:30:17.922 INFO:teuthology.orchestra.run.smithi155.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-02-10T07:30:17.981 INFO:teuthology.orchestra.run.smithi155.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2024-02-10T07:30:18.032 INFO:teuthology.orchestra.run.smithi155.stdout:(4/6): libcephsqlite-19.0.0-1361.g290212a1.el9. 437 kB/s | 163 kB 00:00 2024-02-10T07:30:18.056 INFO:teuthology.orchestra.run.smithi155.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-02-10T07:30:18.072 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 122 kB/s | 19 kB 00:00 2024-02-10T07:30:18.248 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.4 MB/s | 548 kB 00:00 2024-02-10T07:30:18.298 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-02-10T07:30:18.340 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 738 kB/s | 31 kB 00:00 2024-02-10T07:30:18.390 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.4 MB/s | 2.0 MB 00:00 2024-02-10T07:30:18.415 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-02-10T07:30:18.715 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 975 kB/s | 863 kB 00:00 2024-02-10T07:30:18.719 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:18.719 INFO:teuthology.orchestra.run.smithi070.stdout:Total 4.5 MB/s | 6.8 MB 00:01 2024-02-10T07:30:18.737 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:18.753 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:18.753 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:18.923 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:18.923 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:19.433 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:19.500 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-10T07:30:19.534 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-10T07:30:19.581 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-10T07:30:19.611 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-10T07:30:19.710 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-10T07:30:19.845 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-10T07:30:19.939 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-10T07:30:19.983 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-10T07:30:20.012 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-10T07:30:20.041 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 10/12 2024-02-10T07:30:20.282 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 11/12 2024-02-10T07:30:20.318 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T07:30:20.807 INFO:teuthology.orchestra.run.smithi155.stdout:(6/6): ceph-test-19.0.0-1361.g290212a1.el9.x86_ 15 MB/s | 47 MB 00:03 2024-02-10T07:30:20.810 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:20.810 INFO:teuthology.orchestra.run.smithi155.stdout:Total 15 MB/s | 48 MB 00:03 2024-02-10T07:30:20.811 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:30:20.831 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:30:20.831 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 1/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 2/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 3/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-10T07:30:21.098 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-10T07:30:21.099 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-10T07:30:21.099 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-10T07:30:21.231 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:30:21.232 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:30:21.468 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-10T07:30:21.469 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-10T07:30:21.470 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:21.470 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:21.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-10T07:30:21.811 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:30:21.881 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-10T07:30:21.915 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-10T07:30:21.959 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-10T07:30:21.999 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-10T07:30:22.018 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T07:30:22.279 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:07 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:22.440 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 7.4 M 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-10T07:30:22.441 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-02-10T07:30:22.442 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:22.443 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 37 M 2024-02-10T07:30:22.443 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 209 M 2024-02-10T07:30:22.443 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:22.648 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 228 kB/s | 32 kB 00:00 2024-02-10T07:30:22.681 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 541 kB/s | 17 kB 00:00 2024-02-10T07:30:22.723 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 837 kB/s | 34 kB 00:00 2024-02-10T07:30:22.790 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-02-10T07:30:22.965 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-02-10T07:30:23.024 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-02-10T07:30:23.049 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.6 MB/s | 217 kB 00:00 2024-02-10T07:30:23.258 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 802 kB 00:00 2024-02-10T07:30:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-02-10T07:30:23.500 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 811 kB/s | 194 kB 00:00 2024-02-10T07:30:23.725 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-1 6.1 MB/s | 7.4 MB 00:01 2024-02-10T07:30:23.833 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 685 kB/s | 245 kB 00:00 2024-02-10T07:30:23.836 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:23.836 INFO:teuthology.orchestra.run.smithi070.stdout:Total 27 MB/s | 37 MB 00:01 2024-02-10T07:30:23.849 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:23.873 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:23.873 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:24.230 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:24.231 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:24.932 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:24.998 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-10T07:30:25.037 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-10T07:30:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-10T07:30:25.174 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-10T07:30:25.342 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T07:30:25.373 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T07:30:25.449 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-10T07:30:26.079 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-10T07:30:26.184 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-10T07:30:26.524 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 5/6 2024-02-10T07:30:26.600 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T07:30:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-10T07:30:28.424 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-10T07:30:28.449 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T07:30:28.960 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 6/6 2024-02-10T07:30:28.960 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 1/6 2024-02-10T07:30:28.960 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2/6 2024-02-10T07:30:28.960 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-10T07:30:28.960 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-10T07:30:28.960 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-16.el9.x86_64 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-10T07:30:29.405 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-10T07:30:29.406 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:29.406 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:29.733 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph 2024-02-10T07:30:30.327 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:40 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:30.491 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:30.494 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================= 2024-02-10T07:30:30.494 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:30:30.494 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================= 2024-02-10T07:30:30.494 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:30:30.494 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.0.0-1361.g290212a1.el9 ceph 6.6 k 2024-02-10T07:30:30.494 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:30:30.494 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.0.0-1361.g290212a1.el9 ceph 2.1 M 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.0.0-1361.g290212a1.el9 ceph 1.5 M 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 247 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.0.0-1361.g290212a1.el9 ceph 4.7 M 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 17 M 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-10T07:30:30.495 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-10T07:30:30.496 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================= 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout:Install 36 Packages 2024-02-10T07:30:30.497 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:30.498 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 30 M 2024-02-10T07:30:30.498 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 107 M 2024-02-10T07:30:30.498 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:30:31.142 INFO:teuthology.orchestra.run.smithi155.stdout:(1/36): ceph-19.0.0-1361.g290212a1.el9.x86_64.r 34 kB/s | 6.6 kB 00:00 2024-02-10T07:30:31.273 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T07:30:31.273 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 1/12 2024-02-10T07:30:31.273 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-10T07:30:31.273 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-10T07:30:31.273 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-10T07:30:31.273 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T07:30:31.274 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T07:30:31.274 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-10T07:30:31.274 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-10T07:30:31.274 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-10T07:30:31.274 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-10T07:30:31.274 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-10T07:30:31.392 INFO:teuthology.orchestra.run.smithi155.stdout:(2/36): ceph-mds-19.0.0-1361.g290212a1.el9.x86_ 4.8 MB/s | 2.1 MB 00:00 2024-02-10T07:30:31.459 INFO:teuthology.orchestra.run.smithi155.stdout:(3/36): ceph-mgr-19.0.0-1361.g290212a1.el9.x86_ 2.9 MB/s | 1.5 MB 00:00 2024-02-10T07:30:31.526 INFO:teuthology.orchestra.run.smithi155.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1361.g2902 3.6 MB/s | 247 kB 00:00 2024-02-10T07:30:31.651 INFO:teuthology.orchestra.run.smithi155.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-02-10T07:30:31.693 INFO:teuthology.orchestra.run.smithi155.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-10T07:30:31.717 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-10T07:30:31.717 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:31.717 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:31.717 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:31.717 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:31.718 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:31.726 INFO:teuthology.orchestra.run.smithi155.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 949 kB/s | 32 kB 00:00 2024-02-10T07:30:31.760 INFO:teuthology.orchestra.run.smithi155.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-02-10T07:30:31.794 INFO:teuthology.orchestra.run.smithi155.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-02-10T07:30:31.836 INFO:teuthology.orchestra.run.smithi155.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-02-10T07:30:31.911 INFO:teuthology.orchestra.run.smithi155.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-02-10T07:30:31.945 INFO:teuthology.orchestra.run.smithi155.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-02-10T07:30:31.978 INFO:teuthology.orchestra.run.smithi155.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-02-10T07:30:32.004 INFO:teuthology.orchestra.run.smithi155.stdout:(14/36): python3-requests-2.25.1-8.el9.noarch.r 4.4 MB/s | 113 kB 00:00 2024-02-10T07:30:32.037 INFO:teuthology.orchestra.run.smithi155.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-02-10T07:30:32.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-rook 2024-02-10T07:30:32.221 INFO:teuthology.orchestra.run.smithi155.stdout:(16/36): ceph-mon-19.0.0-1361.g290212a1.el9.x86 4.3 MB/s | 4.7 MB 00:01 2024-02-10T07:30:32.238 INFO:teuthology.orchestra.run.smithi155.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-02-10T07:30:32.480 INFO:teuthology.orchestra.run.smithi155.stdout:(18/36): ceph-osd-19.0.0-1361.g290212a1.el9.x86 15 MB/s | 17 MB 00:01 2024-02-10T07:30:32.720 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:17 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:32.881 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:32.882 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================== 2024-02-10T07:30:32.882 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:30:32.882 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================== 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 52 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-10T07:30:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-10T07:30:32.884 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-10T07:30:32.884 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:32.884 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:32.884 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================== 2024-02-10T07:30:32.884 INFO:teuthology.orchestra.run.smithi070.stdout:Install 13 Packages 2024-02-10T07:30:32.884 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:32.884 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.0 M 2024-02-10T07:30:32.885 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 26 M 2024-02-10T07:30:32.885 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:32.888 INFO:teuthology.orchestra.run.smithi155.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 67 kB/s | 43 kB 00:00 2024-02-10T07:30:33.014 INFO:teuthology.orchestra.run.smithi155.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 191 kB/s | 151 kB 00:00 2024-02-10T07:30:33.192 INFO:teuthology.orchestra.run.smithi070.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 170 kB/s | 24 kB 00:00 2024-02-10T07:30:33.208 INFO:teuthology.orchestra.run.smithi070.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 113 kB/s | 18 kB 00:00 2024-02-10T07:30:33.281 INFO:teuthology.orchestra.run.smithi155.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 40 kB/s | 11 kB 00:00 2024-02-10T07:30:33.283 INFO:teuthology.orchestra.run.smithi070.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 174 kB 00:00 2024-02-10T07:30:33.306 INFO:teuthology.orchestra.run.smithi155.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 208 kB/s | 172 kB 00:00 2024-02-10T07:30:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2024-02-10T07:30:33.334 INFO:teuthology.orchestra.run.smithi070.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-10T07:30:33.356 INFO:teuthology.orchestra.run.smithi155.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 767 kB/s | 358 kB 00:00 2024-02-10T07:30:33.359 INFO:teuthology.orchestra.run.smithi070.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 856 kB/s | 43 kB 00:00 2024-02-10T07:30:33.385 INFO:teuthology.orchestra.run.smithi070.stdout:(7/13): ceph-mgr-rook-19.0.0-1361.g290212a1.el9 155 kB/s | 52 kB 00:00 2024-02-10T07:30:33.490 INFO:teuthology.orchestra.run.smithi155.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 85 kB/s | 18 kB 00:00 2024-02-10T07:30:33.507 INFO:teuthology.orchestra.run.smithi155.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 115 kB/s | 23 kB 00:00 2024-02-10T07:30:33.549 INFO:teuthology.orchestra.run.smithi155.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 101 kB/s | 19 kB 00:00 2024-02-10T07:30:33.649 INFO:teuthology.orchestra.run.smithi155.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 124 kB/s | 20 kB 00:00 2024-02-10T07:30:33.707 INFO:teuthology.orchestra.run.smithi155.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 231 kB/s | 46 kB 00:00 2024-02-10T07:30:33.732 INFO:teuthology.orchestra.run.smithi155.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 429 kB/s | 79 kB 00:00 2024-02-10T07:30:33.918 INFO:teuthology.orchestra.run.smithi155.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 78 kB/s | 16 kB 00:00 2024-02-10T07:30:33.963 INFO:teuthology.orchestra.run.smithi155.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 395 kB/s | 90 kB 00:00 2024-02-10T07:30:33.986 INFO:teuthology.orchestra.run.smithi155.stdout:(32/36): python3-pecan-1.4.2-3.el9.noarch.rpm 809 kB/s | 272 kB 00:00 2024-02-10T07:30:34.061 INFO:teuthology.orchestra.run.smithi155.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 251 kB/s | 36 kB 00:00 2024-02-10T07:30:34.220 INFO:teuthology.orchestra.run.smithi155.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 888 kB/s | 230 kB 00:00 2024-02-10T07:30:34.245 INFO:teuthology.orchestra.run.smithi155.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 109 kB/s | 20 kB 00:00 2024-02-10T07:30:34.310 INFO:teuthology.orchestra.run.smithi070.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 16 kB/s | 15 kB 00:00 2024-02-10T07:30:34.343 INFO:teuthology.orchestra.run.smithi070.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 32 kB/s | 32 kB 00:01 2024-02-10T07:30:34.403 INFO:teuthology.orchestra.run.smithi155.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.0 MB/s | 427 kB 00:00 2024-02-10T07:30:34.407 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:34.407 INFO:teuthology.orchestra.run.smithi155.stdout:Total 7.6 MB/s | 30 MB 00:03 2024-02-10T07:30:34.430 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:30:34.473 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:30:34.473 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:30:34.872 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:30:34.873 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:30:34.919 INFO:teuthology.orchestra.run.smithi070.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 132 kB/s | 203 kB 00:01 2024-02-10T07:30:35.302 INFO:teuthology.orchestra.run.smithi070.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 62 kB/s | 59 kB 00:00 2024-02-10T07:30:35.578 INFO:teuthology.orchestra.run.smithi070.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 136 kB/s | 90 kB 00:00 2024-02-10T07:30:35.645 INFO:teuthology.orchestra.run.smithi070.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 783 kB/s | 1.0 MB 00:01 2024-02-10T07:30:35.649 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:35.650 INFO:teuthology.orchestra.run.smithi070.stdout:Total 759 kB/s | 2.0 MB 00:02 2024-02-10T07:30:35.650 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:35.666 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:35.666 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:35.771 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:35.772 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:36.224 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:30:36.290 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-10T07:30:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:36.332 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-10T07:30:36.366 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-10T07:30:36.437 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-10T07:30:36.448 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-10T07:30:36.482 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-10T07:30:36.504 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-10T07:30:36.515 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-10T07:30:36.559 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-10T07:30:36.588 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-10T07:30:36.606 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-10T07:30:36.630 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-10T07:30:36.688 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-10T07:30:36.722 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-10T07:30:36.734 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-10T07:30:36.816 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-10T07:30:36.831 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-10T07:30:36.868 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-10T07:30:36.881 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-10T07:30:36.907 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-10T07:30:36.956 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-10T07:30:37.010 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-10T07:30:37.199 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-10T07:30:37.261 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-10T07:30:37.308 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-10T07:30:37.343 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-10T07:30:37.373 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-10T07:30:37.450 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-10T07:30:37.492 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T07:30:37.525 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-10T07:30:37.565 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-10T07:30:37.598 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-10T07:30:37.600 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-10T07:30:37.635 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-10T07:30:37.653 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-10T07:30:37.687 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-10T07:30:37.725 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-10T07:30:37.757 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-10T07:30:37.767 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-10T07:30:37.782 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T07:30:37.838 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-10T07:30:37.976 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-10T07:30:38.101 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T07:30:38.268 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 31/36 2024-02-10T07:30:38.296 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T07:30:38.320 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 32/36 2024-02-10T07:30:38.321 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:38.321 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-10T07:30:38.321 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T07:30:38.321 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-10T07:30:38.321 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 1/13 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-10T07:30:38.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-10T07:30:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-10T07:30:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-10T07:30:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-10T07:30:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-10T07:30:38.514 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-10T07:30:38.882 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-10T07:30:38.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-10T07:30:38.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-10T07:30:38.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-10T07:30:38.883 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-10T07:30:38.883 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:38.883 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:39.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-cephadm 2024-02-10T07:30:39.726 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:24 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:39.755 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T07:30:39.781 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 33/36 2024-02-10T07:30:39.782 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:39.782 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-10T07:30:39.782 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T07:30:39.782 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-10T07:30:39.782 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:39.887 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================== 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================== 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 138 k 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-10T07:30:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================== 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-02-10T07:30:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:39.890 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 6.8 M 2024-02-10T07:30:39.890 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 33 M 2024-02-10T07:30:39.890 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:40.122 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 227 kB 00:00 2024-02-10T07:30:40.223 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T07:30:40.246 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 34/36 2024-02-10T07:30:40.246 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:40.246 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-10T07:30:40.246 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T07:30:40.246 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-10T07:30:40.246 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:40.255 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-02-10T07:30:40.331 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): ceph-mgr-cephadm-19.0.0-1361.g290212a1.e 376 kB/s | 138 kB 00:00 2024-02-10T07:30:40.457 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T07:30:40.479 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 35/36 2024-02-10T07:30:40.479 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:40.480 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-10T07:30:40.480 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T07:30:40.480 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-10T07:30:40.480 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:40.547 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T07:30:40.714 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 133 kB/s | 51 kB 00:00 2024-02-10T07:30:40.739 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 119 kB/s | 58 kB 00:00 2024-02-10T07:30:41.531 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 389 kB/s | 548 kB 00:01 2024-02-10T07:30:41.533 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:41.533 INFO:teuthology.orchestra.run.smithi070.stdout:Total 4.1 MB/s | 6.8 MB 00:01 2024-02-10T07:30:41.534 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:41.543 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:41.543 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:41.636 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:41.637 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:42.114 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-10T07:30:42.166 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 36/36 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.0.0-1361.g290212a1.el9.x86_64 1/36 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2/36 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 3/36 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 4/36 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 5/36 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9. 6/36 2024-02-10T07:30:42.254 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-10T07:30:42.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-10T07:30:42.256 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-10T07:30:42.256 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-10T07:30:42.256 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-10T07:30:42.256 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-10T07:30:42.256 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-10T07:30:42.256 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-10T07:30:42.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-10T07:30:42.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-10T07:30:42.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-10T07:30:42.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-10T07:30:42.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-10T07:30:42.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-10T07:30:42.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-10T07:30:42.682 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-10T07:30:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-10T07:30:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-10T07:30:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T07:30:43.077 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-10T07:30:43.077 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-10T07:30:43.078 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-10T07:30:43.079 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-10T07:30:43.080 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:43.081 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:43.445 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-base 2024-02-10T07:30:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T07:30:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/6 2024-02-10T07:30:43.655 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-10T07:30:43.656 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-10T07:30:43.656 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-10T07:30:43.656 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-10T07:30:43.982 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-10T07:30:43.982 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:43.982 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:43.982 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:43.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-10T07:30:43.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-10T07:30:43.982 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-10T07:30:43.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-10T07:30:43.983 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-10T07:30:43.983 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:43.983 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:44.027 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:54 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:44.133 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-base-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:44.187 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:44.188 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:30:44.188 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:44.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-fuse 2024-02-10T07:30:44.278 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install cephadm 2024-02-10T07:30:44.806 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:29 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:44.890 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:54 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:44.965 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 885 k 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:44.966 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:44.967 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:44.967 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-02-10T07:30:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:44.967 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 964 k 2024-02-10T07:30:44.967 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 2.8 M 2024-02-10T07:30:44.967 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:45.060 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 748 k 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:30:45.061 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:45.062 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-02-10T07:30:45.062 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:45.062 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 748 k 2024-02-10T07:30:45.062 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 754 k 2024-02-10T07:30:45.062 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:30:45.090 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 637 kB/s | 79 kB 00:00 2024-02-10T07:30:45.291 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): ceph-fuse-19.0.0-1361.g290212a1.el9.x86_ 2.7 MB/s | 885 kB 00:00 2024-02-10T07:30:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:45.292 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.9 MB/s | 964 kB 00:00 2024-02-10T07:30:45.293 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:45.314 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:45.314 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:45.381 INFO:teuthology.orchestra.run.smithi155.stdout:cephadm-19.0.0-1361.g290212a1.el9.noarch.rpm 2.3 MB/s | 748 kB 00:00 2024-02-10T07:30:45.382 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:45.382 INFO:teuthology.orchestra.run.smithi155.stdout:Total 2.3 MB/s | 748 kB 00:00 2024-02-10T07:30:45.383 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:30:45.388 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:30:45.388 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:30:45.390 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:45.390 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:45.394 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:30:45.394 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:30:45.467 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:30:45.576 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-10T07:30:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T07:30:45.839 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:45.937 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:46.340 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:46.449 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T07:30:46.449 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/2 2024-02-10T07:30:46.707 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/1 2024-02-10T07:30:46.708 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:46.708 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:30:46.708 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:46.708 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:46.708 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:46.757 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-10T07:30:46.757 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:46.758 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:46.758 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-10T07:30:46.758 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:46.758 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:46.946 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-immutable-object-cache 2024-02-10T07:30:47.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-volume 2024-02-10T07:30:47.543 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:57 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:47.660 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:32 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:47.703 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:47.704 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:47.704 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-02-10T07:30:47.704 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:47.704 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1361.g290212a1.el9 ceph 145 k 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 145 k 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 436 k 2024-02-10T07:30:47.705 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:30:47.820 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:47.820 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:47.820 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-02-10T07:30:47.820 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:47.820 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 264 k 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-02-10T07:30:47.821 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:47.822 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 332 k 2024-02-10T07:30:47.822 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 1.5 M 2024-02-10T07:30:47.822 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:47.938 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 593 kB/s | 69 kB 00:00 2024-02-10T07:30:47.943 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-immutable-object-cache-19.0.0-1361.g290212 612 kB/s | 145 kB 00:00 2024-02-10T07:30:47.943 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:47.944 INFO:teuthology.orchestra.run.smithi155.stdout:Total 609 kB/s | 145 kB 00:00 2024-02-10T07:30:47.944 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:30:47.951 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:30:47.951 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:30:48.004 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:30:48.004 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:30:48.096 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): ceph-volume-19.0.0-1361.g290212a1.el9.no 961 kB/s | 264 kB 00:00 2024-02-10T07:30:48.097 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:48.098 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.2 MB/s | 332 kB 00:00 2024-02-10T07:30:48.098 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:48.107 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:48.107 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:48.148 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:30:48.150 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:48.151 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:48.219 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T07:30:48.244 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T07:30:48.244 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:48.244 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-10T07:30:48.244 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:48.316 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-10T07:30:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T07:30:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T07:30:48.556 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:30:48.556 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-10T07:30:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:49.250 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1361.g290212a1. 1/1 2024-02-10T07:30:49.250 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:49.250 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:30:49.250 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:49.250 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:49.250 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:49.324 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 1/2 2024-02-10T07:30:49.508 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr 2024-02-10T07:30:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-10T07:30:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:49.672 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-10T07:30:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:49.672 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:50.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados-devel 2024-02-10T07:30:50.106 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:00 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:50.213 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-mgr-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:50.265 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:50.266 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:30:50.267 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:50.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-dashboard 2024-02-10T07:30:50.601 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:35 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:50.762 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:50.763 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:50.763 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 127 k 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 127 k 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 456 k 2024-02-10T07:30:50.764 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:50.945 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:00 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:51.053 INFO:teuthology.orchestra.run.smithi070.stdout:librados-devel-19.0.0-1361.g290212a1.el9.x86_64 439 kB/s | 127 kB 00:00 2024-02-10T07:30:51.054 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:51.054 INFO:teuthology.orchestra.run.smithi070.stdout:Total 437 kB/s | 127 kB 00:00 2024-02-10T07:30:51.054 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:51.059 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:51.059 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:51.104 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:51.105 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-02-10T07:30:51.105 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:30:51.105 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-02-10T07:30:51.105 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:30:51.105 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 1.8 M 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 28 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 15 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:30:51.106 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-02-10T07:30:51.107 INFO:teuthology.orchestra.run.smithi155.stdout:Install 12 Packages 2024-02-10T07:30:51.107 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:51.107 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 6.8 M 2024-02-10T07:30:51.107 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 26 M 2024-02-10T07:30:51.107 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:30:51.115 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:51.115 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:51.255 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:51.360 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:30:51.472 INFO:teuthology.orchestra.run.smithi155.stdout:(1/12): ceph-prometheus-alerts-19.0.0-1361.g290 75 kB/s | 15 kB 00:00 2024-02-10T07:30:51.498 INFO:teuthology.orchestra.run.smithi155.stdout:(2/12): ceph-grafana-dashboards-19.0.0-1361.g29 124 kB/s | 28 kB 00:00 2024-02-10T07:30:51.665 INFO:teuthology.orchestra.run.smithi155.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.4 MB/s | 238 kB 00:00 2024-02-10T07:30:51.723 INFO:teuthology.orchestra.run.smithi155.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-02-10T07:30:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:30:52.032 INFO:teuthology.orchestra.run.smithi155.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1361.g290212a 2.3 MB/s | 1.8 MB 00:00 2024-02-10T07:30:52.154 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:30:52.154 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:52.154 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:52.154 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:52.154 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:52.154 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:52.307 INFO:teuthology.orchestra.run.smithi155.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-02-10T07:30:52.333 INFO:teuthology.orchestra.run.smithi155.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 900 kB/s | 548 kB 00:00 2024-02-10T07:30:52.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs2 2024-02-10T07:30:52.433 INFO:teuthology.orchestra.run.smithi155.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.4 MB/s | 144 kB 00:00 2024-02-10T07:30:52.541 INFO:teuthology.orchestra.run.smithi155.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 284 kB/s | 31 kB 00:00 2024-02-10T07:30:52.642 INFO:teuthology.orchestra.run.smithi155.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2024-02-10T07:30:52.750 INFO:teuthology.orchestra.run.smithi155.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 796 kB/s | 863 kB 00:01 2024-02-10T07:30:52.859 INFO:teuthology.orchestra.run.smithi155.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.7 MB/s | 2.0 MB 00:00 2024-02-10T07:30:52.863 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:52.863 INFO:teuthology.orchestra.run.smithi155.stdout:Total 3.9 MB/s | 6.8 MB 00:01 2024-02-10T07:30:52.883 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:30:52.901 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:30:52.901 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:30:52.980 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:37 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:53.082 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:30:53.083 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:30:53.087 INFO:teuthology.orchestra.run.smithi070.stdout:Package libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:53.139 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:53.141 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:53.141 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:53.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs-devel 2024-02-10T07:30:53.600 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:30:53.676 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-10T07:30:53.720 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-10T07:30:53.766 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-10T07:30:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:38 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:53.805 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-10T07:30:53.890 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-10T07:30:53.957 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 31 k 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 31 k 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 140 k 2024-02-10T07:30:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:30:54.015 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-10T07:30:54.109 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-10T07:30:54.142 INFO:teuthology.orchestra.run.smithi070.stdout:libcephfs-devel-19.0.0-1361.g290212a1.el9.x86_6 170 kB/s | 31 kB 00:00 2024-02-10T07:30:54.142 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:54.143 INFO:teuthology.orchestra.run.smithi070.stdout:Total 169 kB/s | 31 kB 00:00 2024-02-10T07:30:54.143 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:30:54.146 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:30:54.146 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:30:54.152 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-10T07:30:54.171 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:30:54.171 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:30:54.188 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-10T07:30:54.219 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 10/12 2024-02-10T07:30:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:30:54.398 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:30:54.461 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 11/12 2024-02-10T07:30:54.504 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T07:30:54.806 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:30:55.085 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:30:55.085 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:55.085 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:30:55.085 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:30:55.085 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:30:55.085 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:55.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 12/12 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el 1/12 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noa 2/12 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9 3/12 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-10T07:30:55.368 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-10T07:30:55.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-10T07:30:55.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-10T07:30:55.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-10T07:30:55.369 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-10T07:30:55.741 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-10T07:30:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-10T07:30:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-10T07:30:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-10T07:30:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-10T07:30:55.742 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:55.742 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:30:55.819 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:40 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:55.924 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:55.977 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:55.978 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:55.978 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:56.019 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-10T07:30:56.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librbd1 2024-02-10T07:30:56.600 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:06 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:30:56.605 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:41 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:56.711 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:56.760 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:30:56.761 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-02-10T07:30:56.761 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:30:56.761 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-02-10T07:30:56.761 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:30:56.761 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 7.4 M 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-10T07:30:56.762 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:56.763 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:30:56.763 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-02-10T07:30:56.763 INFO:teuthology.orchestra.run.smithi155.stdout:Install 12 Packages 2024-02-10T07:30:56.763 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:30:56.763 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 37 M 2024-02-10T07:30:56.763 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 209 M 2024-02-10T07:30:56.763 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:30:56.764 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:56.765 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:56.765 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:56.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rados 2024-02-10T07:30:57.009 INFO:teuthology.orchestra.run.smithi155.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 254 kB/s | 32 kB 00:00 2024-02-10T07:30:57.034 INFO:teuthology.orchestra.run.smithi155.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 725 kB/s | 17 kB 00:00 2024-02-10T07:30:57.068 INFO:teuthology.orchestra.run.smithi155.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-02-10T07:30:57.118 INFO:teuthology.orchestra.run.smithi155.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-02-10T07:30:57.301 INFO:teuthology.orchestra.run.smithi155.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-02-10T07:30:57.385 INFO:teuthology.orchestra.run.smithi155.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-02-10T07:30:57.406 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:42 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:57.410 INFO:teuthology.orchestra.run.smithi155.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 217 kB 00:00 2024-02-10T07:30:57.513 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:57.563 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:57.564 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:57.564 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:57.602 INFO:teuthology.orchestra.run.smithi155.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 802 kB 00:00 2024-02-10T07:30:57.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rgw 2024-02-10T07:30:57.753 INFO:teuthology.orchestra.run.smithi155.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-02-10T07:30:57.778 INFO:teuthology.orchestra.run.smithi155.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 194 kB 00:00 2024-02-10T07:30:58.012 INFO:teuthology.orchestra.run.smithi155.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-1 6.5 MB/s | 7.4 MB 00:01 2024-02-10T07:30:58.087 INFO:teuthology.orchestra.run.smithi155.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 737 kB/s | 245 kB 00:00 2024-02-10T07:30:58.089 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:58.089 INFO:teuthology.orchestra.run.smithi155.stdout:Total 28 MB/s | 37 MB 00:01 2024-02-10T07:30:58.102 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:30:58.127 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:30:58.127 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:30:58.206 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:43 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:58.313 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:58.364 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:58.365 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:58.365 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:58.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-cephfs 2024-02-10T07:30:58.483 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:30:58.483 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:30:59.007 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:44 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:59.117 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:30:59.169 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:30:59.170 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:30:59.170 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:30:59.194 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:30:59.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rbd 2024-02-10T07:30:59.285 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-10T07:30:59.316 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-10T07:30:59.409 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-10T07:30:59.452 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-10T07:30:59.622 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T07:30:59.660 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T07:30:59.736 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-10T07:30:59.839 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:44 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:30:59.950 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:00.003 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:31:00.004 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-02-10T07:31:00.004 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:31:00.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-fuse 2024-02-10T07:31:00.387 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-10T07:31:00.506 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-10T07:31:00.664 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:45 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:31:00.823 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:31:00.823 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 85 k 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 85 k 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 231 k 2024-02-10T07:31:00.824 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:31:01.053 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-fuse-19.0.0-1361.g290212a1.el9.x86_64.rpm 372 kB/s | 85 kB 00:00 2024-02-10T07:31:01.053 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:01.054 INFO:teuthology.orchestra.run.smithi070.stdout:Total 370 kB/s | 85 kB 00:00 2024-02-10T07:31:01.054 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:31:01.061 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:31:01.061 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:31:01.117 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:31:01.117 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:31:01.265 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:31:01.351 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:01.752 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-10T07:31:01.788 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:02.135 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:02.135 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:02.135 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:31:02.135 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:02.135 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:02.135 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:31:02.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-mirror 2024-02-10T07:31:02.784 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-10T07:31:02.811 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T07:31:02.947 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:47 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:31:03.104 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:31:03.104 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 3.1 M 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 13 M 2024-02-10T07:31:03.105 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:31:03.623 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-mirror-19.0.0-1361.g290212a1.el9.x86_64.rpm 6.0 MB/s | 3.1 MB 00:00 2024-02-10T07:31:03.624 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:03.624 INFO:teuthology.orchestra.run.smithi070.stdout:Total 5.9 MB/s | 3.1 MB 00:00 2024-02-10T07:31:03.624 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:31:03.633 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:31:03.633 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:31:03.706 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:31:03.706 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:31:04.123 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:31:04.190 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:04.215 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:04.215 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:31:04.215 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-10T07:31:04.215 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T07:31:04.215 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T07:31:04.216 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:05.229 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 12/12 2024-02-10T07:31:05.229 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1361.g29021 1/12 2024-02-10T07:31:05.229 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-10T07:31:05.229 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-10T07:31:05.229 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-10T07:31:05.230 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-10T07:31:05.230 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-10T07:31:05.230 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-10T07:31:05.230 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-10T07:31:05.230 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-10T07:31:05.230 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-10T07:31:05.230 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-10T07:31:05.364 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:05.364 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:05.364 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:31:05.364 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:05.364 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:05.365 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:31:05.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-nbd 2024-02-10T07:31:05.667 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-10T07:31:05.667 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:05.667 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:05.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:31:05.667 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-10T07:31:05.667 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-10T07:31:05.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-10T07:31:05.669 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:05.669 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:06.126 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-rook 2024-02-10T07:31:06.186 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:51 ago on Sat 10 Feb 2024 07:29:15 AM UTC. 2024-02-10T07:31:06.344 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 171 k 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-02-10T07:31:06.345 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-02-10T07:31:06.346 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:06.346 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 171 k 2024-02-10T07:31:06.346 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 494 k 2024-02-10T07:31:06.346 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-02-10T07:31:06.594 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-nbd-19.0.0-1361.g290212a1.el9.x86_64.rpm 690 kB/s | 171 kB 00:00 2024-02-10T07:31:06.594 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:06.594 INFO:teuthology.orchestra.run.smithi070.stdout:Total 685 kB/s | 171 kB 00:00 2024-02-10T07:31:06.595 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-02-10T07:31:06.602 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-02-10T07:31:06.602 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-02-10T07:31:06.656 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-02-10T07:31:06.657 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-02-10T07:31:06.722 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:16 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:06.801 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-02-10T07:31:06.882 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:06.883 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-02-10T07:31:06.883 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:31:06.883 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 52 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-10T07:31:06.884 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout:Install 13 Packages 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:06.885 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 2.0 M 2024-02-10T07:31:06.886 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 26 M 2024-02-10T07:31:06.886 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:06.900 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:07.178 INFO:teuthology.orchestra.run.smithi155.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 153 kB/s | 18 kB 00:00 2024-02-10T07:31:07.195 INFO:teuthology.orchestra.run.smithi155.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 181 kB/s | 24 kB 00:00 2024-02-10T07:31:07.246 INFO:teuthology.orchestra.run.smithi155.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 174 kB 00:00 2024-02-10T07:31:07.271 INFO:teuthology.orchestra.run.smithi155.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-10T07:31:07.296 INFO:teuthology.orchestra.run.smithi155.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-10T07:31:07.321 INFO:teuthology.orchestra.run.smithi155.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 855 kB/s | 43 kB 00:00 2024-02-10T07:31:07.338 INFO:teuthology.orchestra.run.smithi155.stdout:(7/13): ceph-mgr-rook-19.0.0-1361.g290212a1.el9 188 kB/s | 52 kB 00:00 2024-02-10T07:31:07.352 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:07.649 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:07.650 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:07.650 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-02-10T07:31:07.650 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:07.650 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:31:07.650 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-02-10T07:31:07.880 DEBUG:teuthology.parallel:result is None 2024-02-10T07:31:08.214 INFO:teuthology.orchestra.run.smithi155.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 17 kB/s | 15 kB 00:00 2024-02-10T07:31:08.322 INFO:teuthology.orchestra.run.smithi155.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 31 kB/s | 32 kB 00:01 2024-02-10T07:31:08.889 INFO:teuthology.orchestra.run.smithi155.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 131 kB/s | 203 kB 00:01 2024-02-10T07:31:09.148 INFO:teuthology.orchestra.run.smithi155.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 72 kB/s | 59 kB 00:00 2024-02-10T07:31:09.556 INFO:teuthology.orchestra.run.smithi155.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 134 kB/s | 90 kB 00:00 2024-02-10T07:31:09.589 INFO:teuthology.orchestra.run.smithi155.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 760 kB/s | 1.0 MB 00:01 2024-02-10T07:31:09.595 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:09.595 INFO:teuthology.orchestra.run.smithi155.stdout:Total 774 kB/s | 2.0 MB 00:02 2024-02-10T07:31:09.596 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:09.610 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:09.611 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:09.721 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:09.723 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:10.247 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:10.411 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-10T07:31:10.482 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-10T07:31:10.546 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-10T07:31:10.584 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-10T07:31:10.625 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-10T07:31:10.720 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-10T07:31:10.828 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-10T07:31:10.877 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-10T07:31:11.605 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-10T07:31:11.658 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-10T07:31:11.691 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-10T07:31:11.748 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-10T07:31:11.770 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 13/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 1/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-10T07:31:12.626 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-10T07:31:12.627 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-10T07:31:12.627 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-10T07:31:12.627 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-10T07:31:12.627 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-10T07:31:13.014 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-10T07:31:13.014 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:13.014 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-10T07:31:13.015 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-10T07:31:13.016 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:13.016 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:13.286 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-cephadm 2024-02-10T07:31:13.868 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:23 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:14.027 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:14.027 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-02-10T07:31:14.027 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 138 k 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:14.028 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-02-10T07:31:14.029 INFO:teuthology.orchestra.run.smithi155.stdout:Install 6 Packages 2024-02-10T07:31:14.029 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:14.029 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 6.8 M 2024-02-10T07:31:14.029 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 33 M 2024-02-10T07:31:14.029 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:14.251 INFO:teuthology.orchestra.run.smithi155.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.5 MB/s | 227 kB 00:00 2024-02-10T07:31:14.410 INFO:teuthology.orchestra.run.smithi155.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-02-10T07:31:14.460 INFO:teuthology.orchestra.run.smithi155.stdout:(3/6): ceph-mgr-cephadm-19.0.0-1361.g290212a1.e 386 kB/s | 138 kB 00:00 2024-02-10T07:31:14.968 INFO:teuthology.orchestra.run.smithi155.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 108 kB/s | 58 kB 00:00 2024-02-10T07:31:14.986 INFO:teuthology.orchestra.run.smithi155.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 97 kB/s | 51 kB 00:00 2024-02-10T07:31:15.536 INFO:teuthology.orchestra.run.smithi155.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 427 kB/s | 548 kB 00:01 2024-02-10T07:31:15.540 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:15.540 INFO:teuthology.orchestra.run.smithi155.stdout:Total 4.5 MB/s | 6.8 MB 00:01 2024-02-10T07:31:15.540 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:15.550 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:15.550 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:15.642 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:15.655 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:15.947 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:16.110 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-10T07:31:16.162 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-10T07:31:16.677 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-10T07:31:16.756 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-10T07:31:16.827 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-10T07:31:16.850 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T07:31:17.604 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 6/6 2024-02-10T07:31:17.605 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 1/6 2024-02-10T07:31:17.605 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-10T07:31:17.605 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-10T07:31:17.605 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-10T07:31:17.605 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-10T07:31:17.943 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-10T07:31:17.943 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:17.943 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:17.943 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:31:17.943 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-10T07:31:17.944 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-10T07:31:17.944 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-10T07:31:17.944 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-10T07:31:17.944 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-10T07:31:17.944 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:17.944 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:18.188 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-fuse 2024-02-10T07:31:18.790 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:28 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:18.950 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 885 k 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:18.951 INFO:teuthology.orchestra.run.smithi155.stdout:Install 2 Packages 2024-02-10T07:31:18.952 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:18.952 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 964 k 2024-02-10T07:31:18.952 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 2.8 M 2024-02-10T07:31:18.952 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:19.072 INFO:teuthology.orchestra.run.smithi155.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 659 kB/s | 79 kB 00:00 2024-02-10T07:31:19.297 INFO:teuthology.orchestra.run.smithi155.stdout:(2/2): ceph-fuse-19.0.0-1361.g290212a1.el9.x86_ 2.5 MB/s | 885 kB 00:00 2024-02-10T07:31:19.298 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:19.298 INFO:teuthology.orchestra.run.smithi155.stdout:Total 2.7 MB/s | 964 kB 00:00 2024-02-10T07:31:19.298 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:19.319 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:19.319 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:19.394 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:19.395 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:19.571 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:19.707 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-10T07:31:19.788 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T07:31:20.530 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2/2 2024-02-10T07:31:20.531 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/2 2024-02-10T07:31:20.872 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-10T07:31:20.872 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:20.872 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:20.872 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-10T07:31:20.872 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:20.872 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:21.162 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-volume 2024-02-10T07:31:21.769 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:31 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:21.928 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.0.0-1361.g290212a1.el9 ceph-noarch 264 k 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-10T07:31:21.929 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:21.930 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:21.930 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:21.930 INFO:teuthology.orchestra.run.smithi155.stdout:Install 2 Packages 2024-02-10T07:31:21.930 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:21.930 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 332 k 2024-02-10T07:31:21.930 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 1.5 M 2024-02-10T07:31:21.930 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:22.044 INFO:teuthology.orchestra.run.smithi155.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 602 kB/s | 69 kB 00:00 2024-02-10T07:31:22.211 INFO:teuthology.orchestra.run.smithi155.stdout:(2/2): ceph-volume-19.0.0-1361.g290212a1.el9.no 938 kB/s | 264 kB 00:00 2024-02-10T07:31:22.212 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:22.213 INFO:teuthology.orchestra.run.smithi155.stdout:Total 1.1 MB/s | 332 kB 00:00 2024-02-10T07:31:22.213 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:22.221 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:22.221 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:22.260 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:22.260 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:22.410 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:22.595 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-10T07:31:22.639 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T07:31:22.654 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2/2 2024-02-10T07:31:22.654 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:31:22.654 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-10T07:31:22.654 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:23.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 1/2 2024-02-10T07:31:23.769 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-10T07:31:23.769 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:23.769 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:23.769 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.0.0-1361.g290212a1.el9.noarch 2024-02-10T07:31:23.769 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-10T07:31:23.769 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:23.769 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:24.129 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados-devel 2024-02-10T07:31:24.725 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:34 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:24.896 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:24.897 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:24.897 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-02-10T07:31:24.897 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:24.897 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 127 k 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 127 k 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 456 k 2024-02-10T07:31:24.898 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:25.160 INFO:teuthology.orchestra.run.smithi155.stdout:librados-devel-19.0.0-1361.g290212a1.el9.x86_64 485 kB/s | 127 kB 00:00 2024-02-10T07:31:25.160 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:25.160 INFO:teuthology.orchestra.run.smithi155.stdout:Total 482 kB/s | 127 kB 00:00 2024-02-10T07:31:25.160 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:25.166 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:25.166 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:25.221 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:25.222 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:25.375 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:25.476 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:25.972 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:26.303 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:26.303 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:26.303 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:26.303 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:26.303 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:26.303 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:26.526 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs2 2024-02-10T07:31:27.134 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:37 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:27.241 INFO:teuthology.orchestra.run.smithi155.stdout:Package libcephfs2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:27.294 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:27.295 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:31:27.295 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:27.376 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs-devel 2024-02-10T07:31:27.964 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:37 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:28.127 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:28.128 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:28.128 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-02-10T07:31:28.128 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:28.128 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.0.0-1361.g290212a1.el9 ceph 31 k 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 31 k 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 140 k 2024-02-10T07:31:28.129 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:28.315 INFO:teuthology.orchestra.run.smithi155.stdout:libcephfs-devel-19.0.0-1361.g290212a1.el9.x86_6 167 kB/s | 31 kB 00:00 2024-02-10T07:31:28.316 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:28.316 INFO:teuthology.orchestra.run.smithi155.stdout:Total 166 kB/s | 31 kB 00:00 2024-02-10T07:31:28.316 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:28.320 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:28.320 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:28.352 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:28.353 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:28.496 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:28.608 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:28.981 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:29.291 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:29.292 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:29.292 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:29.292 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:29.292 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:29.421 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados2 2024-02-10T07:31:30.011 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:40 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:30.116 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:30.168 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:30.169 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:31:30.169 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:30.216 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librbd1 2024-02-10T07:31:30.798 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:40 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:30.905 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:30.956 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:30.957 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:31:30.957 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:31.006 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rados 2024-02-10T07:31:31.597 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:41 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:31.703 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rados-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:31.754 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:31.756 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:31:31.756 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:31.810 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rgw 2024-02-10T07:31:32.403 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:42 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:32.509 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rgw-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:32.560 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:32.561 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:31:32.561 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:32.606 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-cephfs 2024-02-10T07:31:33.192 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:43 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:33.301 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-cephfs-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:33.354 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:33.355 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:31:33.355 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:33.402 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rbd 2024-02-10T07:31:33.993 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:43 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:34.099 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rbd-2:19.0.0-1361.g290212a1.el9.x86_64 is already installed. 2024-02-10T07:31:34.150 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:34.151 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-02-10T07:31:34.151 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:34.215 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-fuse 2024-02-10T07:31:34.798 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:44 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:34.956 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:34.957 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:34.957 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-02-10T07:31:34.957 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:34.957 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.0.0-1361.g290212a1.el9 ceph 85 k 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 85 k 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 231 k 2024-02-10T07:31:34.958 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:35.210 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-fuse-19.0.0-1361.g290212a1.el9.x86_64.rpm 339 kB/s | 85 kB 00:00 2024-02-10T07:31:35.211 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:35.211 INFO:teuthology.orchestra.run.smithi155.stdout:Total 336 kB/s | 85 kB 00:00 2024-02-10T07:31:35.211 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:35.218 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:35.218 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:35.274 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:35.274 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:35.424 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:35.517 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:35.922 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:36.257 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:36.257 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:36.257 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:36.257 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:36.258 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:36.258 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:36.487 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-mirror 2024-02-10T07:31:37.092 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:47 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:37.253 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:37.253 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.0.0-1361.g290212a1.el9 ceph 3.1 M 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-02-10T07:31:37.254 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:37.255 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 3.1 M 2024-02-10T07:31:37.255 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 13 M 2024-02-10T07:31:37.255 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:37.764 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-mirror-19.0.0-1361.g290212a1.el9.x86_64.rpm 6.1 MB/s | 3.1 MB 00:00 2024-02-10T07:31:37.764 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:37.765 INFO:teuthology.orchestra.run.smithi155.stdout:Total 6.0 MB/s | 3.1 MB 00:00 2024-02-10T07:31:37.765 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:37.774 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:37.774 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:37.847 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:37.847 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:38.272 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:38.340 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:38.364 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:38.364 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-10T07:31:38.365 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-10T07:31:38.365 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T07:31:38.365 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-10T07:31:38.365 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:39.499 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:39.499 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:39.499 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:39.499 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:39.499 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:39.499 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:39.743 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-nbd 2024-02-10T07:31:40.343 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:50 ago on Sat 10 Feb 2024 07:29:50 AM UTC. 2024-02-10T07:31:40.502 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-02-10T07:31:40.502 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:40.502 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-02-10T07:31:40.502 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:40.502 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-02-10T07:31:40.502 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.0.0-1361.g290212a1.el9 ceph 171 k 2024-02-10T07:31:40.503 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:40.503 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-02-10T07:31:40.503 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-02-10T07:31:40.503 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-02-10T07:31:40.503 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:40.503 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 171 k 2024-02-10T07:31:40.503 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 494 k 2024-02-10T07:31:40.504 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-02-10T07:31:40.751 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-nbd-19.0.0-1361.g290212a1.el9.x86_64.rpm 689 kB/s | 171 kB 00:00 2024-02-10T07:31:40.752 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:40.752 INFO:teuthology.orchestra.run.smithi155.stdout:Total 685 kB/s | 171 kB 00:00 2024-02-10T07:31:40.752 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-02-10T07:31:40.760 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-02-10T07:31:40.760 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-02-10T07:31:40.815 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-02-10T07:31:40.815 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-02-10T07:31:40.963 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-02-10T07:31:41.058 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:41.532 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:41.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 1/1 2024-02-10T07:31:41.840 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:41.840 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-02-10T07:31:41.840 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.0.0-1361.g290212a1.el9.x86_64 2024-02-10T07:31:41.840 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:31:41.840 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-02-10T07:31:42.072 DEBUG:teuthology.parallel:result is None 2024-02-10T07:31:42.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:31:42.229 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-10T07:31:42.254 INFO:teuthology.orchestra.run.smithi070.stdout:19.0.0-1361.g290212a1.el9 2024-02-10T07:31:42.255 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1361.g290212a1.el9 2024-02-10T07:31:42.255 INFO:teuthology.task.install:The correct ceph version 19.0.0-1361.g290212a1 is installed. 2024-02-10T07:31:42.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:31:42.425 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-10T07:31:42.447 INFO:teuthology.orchestra.run.smithi155.stdout:19.0.0-1361.g290212a1.el9 2024-02-10T07:31:42.447 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1361.g290212a1.el9 2024-02-10T07:31:42.447 INFO:teuthology.task.install:The correct ceph version 19.0.0-1361.g290212a1 is installed. 2024-02-10T07:31:42.449 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-10T07:31:42.449 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:31:42.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-10T07:31:42.482 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:31:42.482 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-10T07:31:42.516 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-10T07:31:42.516 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:31:42.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-02-10T07:31:42.552 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-10T07:31:42.619 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:31:42.620 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-02-10T07:31:42.650 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-10T07:31:42.718 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-10T07:31:42.718 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:31:42.718 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-10T07:31:42.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-10T07:31:42.812 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:31:42.812 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-10T07:31:42.839 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-10T07:31:42.906 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-10T07:31:42.906 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:31:42.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-02-10T07:31:42.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-10T07:31:43.003 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:31:43.003 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-02-10T07:31:43.029 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-10T07:31:43.096 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-10T07:31:43.183 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'mon_bind_msgr2': False, 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'cephadm_mode': 'root'} 2024-02-10T07:31:43.183 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:31:43.184 INFO:tasks.cephadm:Cluster fsid is 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:31:43.184 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-10T07:31:43.184 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.70:6789]', 'mon.c': '[v1:172.21.15.70:6790]', 'mon.b': '[v1:172.21.15.155:6789]'} 2024-02-10T07:31:43.184 INFO:tasks.cephadm:First mon is mon.a on smithi070 2024-02-10T07:31:43.184 INFO:tasks.cephadm:First mgr is y 2024-02-10T07:31:43.184 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-10T07:31:43.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2024-02-10T07:31:43.211 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-02-10T07:31:43.238 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-10T07:31:43.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:31:43.388 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76638/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1361-g290212a1', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1361.g290212a1'}, 'url': 'https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-09 18:34:21.832169', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-10T07:31:43.536 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref main, sha1 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:31:43.538 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm 2024-02-10T07:31:43.538 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T07:31:43.946 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 07:31 /home/ubuntu/cephtest/cephadm 2024-02-10T07:31:43.947 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T07:31:44.384 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 07:31 /home/ubuntu/cephtest/cephadm 2024-02-10T07:31:44.384 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T07:31:44.401 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T07:31:44.432 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 on all hosts... 2024-02-10T07:31:44.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T07:31:44.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T07:31:45.248 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:31:45.355 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:32:17.263 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-02-10T07:32:17.263 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:32:17.263 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T07:32:17.263 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-02-10T07:32:17.263 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T07:32:17.263 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-02-10T07:32:17.264 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-02-10T07:32:17.705 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-02-10T07:32:17.706 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:32:17.706 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T07:32:17.706 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2024-02-10T07:32:17.706 INFO:teuthology.orchestra.run.smithi070.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T07:32:17.706 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-02-10T07:32:17.706 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-02-10T07:32:17.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2024-02-10T07:32:17.790 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-02-10T07:32:17.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2024-02-10T07:32:17.848 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-02-10T07:32:17.890 INFO:tasks.cephadm:Writing seed config... 2024-02-10T07:32:17.891 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-10T07:32:17.891 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-10T07:32:17.891 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-10T07:32:17.891 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-02-10T07:32:17.892 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-10T07:32:17.893 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:32:17.893 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-10T07:32:17.907 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 70142e68-c7e6-11ee-95b8-87774f69a715 mon election default strategy = 3 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-10T07:32:17.908 DEBUG:teuthology.orchestra.run.smithi070:mon.a> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a.service 2024-02-10T07:32:17.950 DEBUG:teuthology.orchestra.run.smithi070:mgr.y> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.y.service 2024-02-10T07:32:17.993 INFO:tasks.cephadm:Bootstrapping... 2024-02-10T07:32:17.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -v bootstrap --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.70:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:32:18.183 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:32:18.183 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', '-v', 'bootstrap', '--fsid', '70142e68-c7e6-11ee-95b8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.70:6789]', '--skip-admin-label'] 2024-02-10T07:32:18.207 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T07:32:18.208 INFO:teuthology.orchestra.run.smithi070.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-10T07:32:18.208 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying podman|docker is present... 2024-02-10T07:32:18.231 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T07:32:18.231 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying lvm2 is present... 2024-02-10T07:32:18.231 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying time synchronization is in place... 2024-02-10T07:32:18.238 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T07:32:18.238 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T07:32:18.244 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T07:32:18.244 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-02-10T07:32:18.250 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-02-10T07:32:18.256 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-02-10T07:32:18.256 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-02-10T07:32:18.256 INFO:teuthology.orchestra.run.smithi070.stdout:Repeating the final host check... 2024-02-10T07:32:18.279 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T07:32:18.279 INFO:teuthology.orchestra.run.smithi070.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-10T07:32:18.279 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl is present 2024-02-10T07:32:18.279 INFO:teuthology.orchestra.run.smithi070.stdout:lvcreate is present 2024-02-10T07:32:18.286 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T07:32:18.286 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T07:32:18.291 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T07:32:18.292 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-02-10T07:32:18.298 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-02-10T07:32:18.304 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-02-10T07:32:18.304 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-02-10T07:32:18.304 INFO:teuthology.orchestra.run.smithi070.stdout:Host looks OK 2024-02-10T07:32:18.304 INFO:teuthology.orchestra.run.smithi070.stdout:Cluster fsid: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:18.304 INFO:teuthology.orchestra.run.smithi070.stdout:Acquiring lock 139792469317232 on /run/cephadm/70142e68-c7e6-11ee-95b8-87774f69a715.lock 2024-02-10T07:32:18.304 INFO:teuthology.orchestra.run.smithi070.stdout:Lock 139792469317232 acquired on /run/cephadm/70142e68-c7e6-11ee-95b8-87774f69a715.lock 2024-02-10T07:32:18.305 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 6789 ... 2024-02-10T07:32:18.305 INFO:teuthology.orchestra.run.smithi070.stdout:Base mon IP(s) is [172.21.15.70:6789], mon addrv is [v1:172.21.15.70:6789] 2024-02-10T07:32:18.308 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.70 metric 100 2024-02-10T07:32:18.308 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.70 metric 100 2024-02-10T07:32:18.310 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-10T07:32:18.310 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-10T07:32:18.310 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1789sec hoplimit 64 pref medium 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2024-02-10T07:32:18.313 INFO:teuthology.orchestra.run.smithi070.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-02-10T07:32:18.314 INFO:teuthology.orchestra.run.smithi070.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-10T07:32:18.314 INFO:teuthology.orchestra.run.smithi070.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T07:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Getting image source signatures 2024-02-10T07:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-10T07:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:5fe0e97a267976e17c7dcb240cf9455472de6876101ae78af4bdf182158b6d8d 2024-02-10T07:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying config sha256:c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T07:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-10T07:32:19.839 INFO:teuthology.orchestra.run.smithi070.stdout:ceph: stdout ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T07:32:19.839 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph version: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T07:32:19.839 INFO:teuthology.orchestra.run.smithi070.stdout:Extracting ceph user uid/gid from container image... 2024-02-10T07:32:20.744 INFO:teuthology.orchestra.run.smithi070.stdout:stat: stdout 167 167 2024-02-10T07:32:20.744 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial keys... 2024-02-10T07:32:21.700 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQCFJsdlqgIuAxAAghegDlPMtXsQxgnI3GBlaQ== 2024-02-10T07:32:22.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQCGJsdlWWEnAhAAnIvYIlNuQPUAkmh+YSN2lw== 2024-02-10T07:32:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQCHJsdlnQZlBRAAc575RG3Kv0o4a5QAtZ1Xyg== 2024-02-10T07:32:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial monmap... 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool for a [v1:172.21.15.70:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = quincy 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: set fsid to 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:24.593 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T07:32:24.594 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:32:24.594 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mon... 2024-02-10T07:32:25.558 INFO:teuthology.orchestra.run.smithi070.stdout:create mon.a on 2024-02-10T07:32:25.754 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-10T07:32:25.893 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-10T07:32:26.047 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-70142e68-c7e6-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-70142e68-c7e6-11ee-95b8-87774f69a715.target. 2024-02-10T07:32:26.047 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-70142e68-c7e6-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-70142e68-c7e6-11ee-95b8-87774f69a715.target. 2024-02-10T07:32:26.259 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a 2024-02-10T07:32:26.260 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a.service: Unit ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a.service not loaded. 2024-02-10T07:32:26.415 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-70142e68-c7e6-11ee-95b8-87774f69a715.target.wants/ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-70142e68-c7e6-11ee-95b8-87774f69a715@.service. 2024-02-10T07:32:26.940 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:32:26.940 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-02-10T07:32:26.946 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:32:26.946 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-02-10T07:32:26.946 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-02-10T07:32:26.946 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T07:32:26.946 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon to start... 2024-02-10T07:32:26.946 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon... 2024-02-10T07:32:28.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout cluster: 2024-02-10T07:32:28.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout id: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:28.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-10T07:32:28.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:28.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout services: 2024-02-10T07:32:28.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.530708s) 2024-02-10T07:32:28.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-10T07:32:28.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout data: 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pgs: 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:28.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.354+0000 7fbfcacbe700 1 Processor -- start 2024-02-10T07:32:28.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.354+0000 7fbfcacbe700 1 -- start start 2024-02-10T07:32:28.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.354+0000 7fbfcacbe700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbfc4078e90 con 0x7fbfc4106560 2024-02-10T07:32:28.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc8a5a700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbfc4106560 0x7fbfc4108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60084/0 (socket says 172.21.15.70:60084) 2024-02-10T07:32:28.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc8a5a700 1 -- 172.21.15.70:0/2004098908 learned_addr learned my addr 172.21.15.70:0/2004098908 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:28.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc37fe700 1 -- 172.21.15.70:0/2004098908 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766139993 0 0) 0x7fbfc4078e90 con 0x7fbfc4106560 2024-02-10T07:32:28.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc37fe700 1 -- 172.21.15.70:0/2004098908 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb40035b0 con 0x7fbfc4106560 2024-02-10T07:32:28.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc37fe700 1 -- 172.21.15.70:0/2004098908 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1778055090 0 0) 0x7fbfb40035b0 con 0x7fbfc4106560 2024-02-10T07:32:28.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc37fe700 1 -- 172.21.15.70:0/2004098908 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc410b080 con 0x7fbfc4106560 2024-02-10T07:32:28.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc37fe700 1 -- 172.21.15.70:0/2004098908 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbfb8001580 con 0x7fbfc4106560 2024-02-10T07:32:28.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc37fe700 1 -- 172.21.15.70:0/2004098908 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbfb80018e0 con 0x7fbfc4106560 2024-02-10T07:32:28.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.355+0000 7fbfc37fe700 1 -- 172.21.15.70:0/2004098908 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbfb8001c20 con 0x7fbfc4106560 2024-02-10T07:32:28.008 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.356+0000 7fbfcacbe700 1 -- 172.21.15.70:0/2004098908 >> v1:172.21.15.70:6789/0 conn(0x7fbfc4106560 legacy=0x7fbfc4108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.356+0000 7fbfcacbe700 1 -- 172.21.15.70:0/2004098908 shutdown_connections 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.356+0000 7fbfcacbe700 1 -- 172.21.15.70:0/2004098908 >> 172.21.15.70:0/2004098908 conn(0x7fbfc40fff70 msgr2=0x7fbfc4102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.356+0000 7fbfcacbe700 1 -- 172.21.15.70:0/2004098908 shutdown_connections 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.356+0000 7fbfcacbe700 1 -- 172.21.15.70:0/2004098908 wait complete. 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfcacbe700 1 Processor -- start 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfcacbe700 1 -- start start 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfcacbe700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbfc419ed00 con 0x7fbfc4106560 2024-02-10T07:32:28.009 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfc8a5a700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbfc4106560 0x7fbfc4105de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60088/0 (socket says 172.21.15.70:60088) 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfc8a5a700 1 -- 172.21.15.70:0/1396660244 learned_addr learned my addr 172.21.15.70:0/1396660244 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531343541 0 0) 0x7fbfc419ed00 con 0x7fbfc4106560 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb40031f0 con 0x7fbfc4106560 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3825660499 0 0) 0x7fbfb40031f0 con 0x7fbfc4106560 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbfc419ed00 con 0x7fbfc4106560 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.357+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbfb8001480 con 0x7fbfc4106560 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3723409745 0 0) 0x7fbfc419ed00 con 0x7fbfc4106560 2024-02-10T07:32:28.010 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc419fea0 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc419eef0 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbfb8004c80 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbfb8004e70 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fbfb8005460 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc419f3f0 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.358+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7fbfb8005c80 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.359+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc410a1b0 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.361+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fbfb8006820 con 0x7fbfc4106560 2024-02-10T07:32:28.011 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.435+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fbfc41921f0 con 0x7fbfc4106560 2024-02-10T07:32:28.012 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.436+0000 7fbfc1ffb700 1 -- 172.21.15.70:0/1396660244 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (unknown 1155462804 0 222860570) 0x7fbfb8006a70 con 0x7fbfc4106560 2024-02-10T07:32:28.012 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.437+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 >> v1:172.21.15.70:6789/0 conn(0x7fbfc4106560 legacy=0x7fbfc4105de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:28.012 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.437+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 shutdown_connections 2024-02-10T07:32:28.012 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.437+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 >> 172.21.15.70:0/1396660244 conn(0x7fbfc40fff70 msgr2=0x7fbfc4107240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:28.012 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.437+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 shutdown_connections 2024-02-10T07:32:28.012 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:27.437+0000 7fbfcacbe700 1 -- 172.21.15.70:0/1396660244 wait complete. 2024-02-10T07:32:28.012 INFO:teuthology.orchestra.run.smithi070.stdout:mon is available 2024-02-10T07:32:28.013 INFO:teuthology.orchestra.run.smithi070.stdout:Assimilating anything we can from ceph.conf... 2024-02-10T07:32:29.014 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:29.014 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T07:32:29.014 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:29.014 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.70:6789] 2024-02-10T07:32:29.014 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.450+0000 7fa44af9b700 1 Processor -- start 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.450+0000 7fa44af9b700 1 -- start start 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa44af9b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa444109f30 con 0x7fa444105b00 2024-02-10T07:32:29.015 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa448d37700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa444105b00 0x7fa444105f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60096/0 (socket says 172.21.15.70:60096) 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa448d37700 1 -- 172.21.15.70:0/2415338669 learned_addr learned my addr 172.21.15.70:0/2415338669 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa4437fe700 1 -- 172.21.15.70:0/2415338669 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318951966 0 0) 0x7fa444109f30 con 0x7fa444105b00 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa4437fe700 1 -- 172.21.15.70:0/2415338669 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa42c0035b0 con 0x7fa444105b00 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa4437fe700 1 -- 172.21.15.70:0/2415338669 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560368012 0 0) 0x7fa42c0035b0 con 0x7fa444105b00 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa4437fe700 1 -- 172.21.15.70:0/2415338669 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44410b0d0 con 0x7fa444105b00 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.451+0000 7fa4437fe700 1 -- 172.21.15.70:0/2415338669 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa43c001900 con 0x7fa444105b00 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.452+0000 7fa4437fe700 1 -- 172.21.15.70:0/2415338669 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa43c001bf0 con 0x7fa444105b00 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.452+0000 7fa4437fe700 1 -- 172.21.15.70:0/2415338669 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa43c001f30 con 0x7fa444105b00 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.452+0000 7fa44af9b700 1 -- 172.21.15.70:0/2415338669 >> v1:172.21.15.70:6789/0 conn(0x7fa444105b00 legacy=0x7fa444105f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.452+0000 7fa44af9b700 1 -- 172.21.15.70:0/2415338669 shutdown_connections 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.452+0000 7fa44af9b700 1 -- 172.21.15.70:0/2415338669 >> 172.21.15.70:0/2415338669 conn(0x7fa4441010e0 msgr2=0x7fa444103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.452+0000 7fa44af9b700 1 -- 172.21.15.70:0/2415338669 shutdown_connections 2024-02-10T07:32:29.016 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.452+0000 7fa44af9b700 1 -- 172.21.15.70:0/2415338669 wait complete. 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa44af9b700 1 Processor -- start 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa44af9b700 1 -- start start 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa44af9b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa4441993d0 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa448d37700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa444105b00 0x7fa444198cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60110/0 (socket says 172.21.15.70:60110) 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa448d37700 1 -- 172.21.15.70:0/3481915391 learned_addr learned my addr 172.21.15.70:0/3481915391 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241978882 0 0) 0x7fa4441993d0 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa42c0031f0 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3898234046 0 0) 0x7fa42c0031f0 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa4441993d0 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.453+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa43c0018c0 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4235183403 0 0) 0x7fa4441993d0 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444199560 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa444199810 con 0x7fa444105b00 2024-02-10T07:32:29.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa43c004ea0 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa43c0051a0 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fa43c005750 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44419d360 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.454+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7fa43c005f20 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.455+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa444192e90 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.457+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fa43c006ad0 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.525+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa44410a200 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.528+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+317 (unknown 1213389831 0 213377013) 0x7fa43c006210 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.528+0000 7fa441ffb700 1 -- 172.21.15.70:0/3481915391 <== mon.0 v1:172.21.15.70:6789/0 11 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fa43c0063b0 con 0x7fa444105b00 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.529+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 >> v1:172.21.15.70:6789/0 conn(0x7fa444105b00 legacy=0x7fa444198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.529+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 shutdown_connections 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.529+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 >> 172.21.15.70:0/3481915391 conn(0x7fa4441010e0 msgr2=0x7fa44418ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:29.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.529+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 shutdown_connections 2024-02-10T07:32:29.019 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:28.529+0000 7fa44af9b700 1 -- 172.21.15.70:0/3481915391 wait complete. 2024-02-10T07:32:29.019 INFO:teuthology.orchestra.run.smithi070.stdout:Generating new minimal ceph.conf... 2024-02-10T07:32:30.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.379+0000 7ff14ffed700 1 Processor -- start 2024-02-10T07:32:30.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.379+0000 7ff14ffed700 1 -- start start 2024-02-10T07:32:30.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.379+0000 7ff14ffed700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff148109f30 con 0x7ff148105b00 2024-02-10T07:32:30.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.379+0000 7ff14dd89700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff148105b00 0x7ff148105f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60118/0 (socket says 172.21.15.70:60118) 2024-02-10T07:32:30.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.379+0000 7ff14dd89700 1 -- 172.21.15.70:0/1129456456 learned_addr learned my addr 172.21.15.70:0/1129456456 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:30.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.380+0000 7ff14cd87700 1 -- 172.21.15.70:0/1129456456 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700142031 0 0) 0x7ff148109f30 con 0x7ff148105b00 2024-02-10T07:32:30.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.380+0000 7ff14cd87700 1 -- 172.21.15.70:0/1129456456 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1340035b0 con 0x7ff148105b00 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.380+0000 7ff14cd87700 1 -- 172.21.15.70:0/1129456456 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328609685 0 0) 0x7ff1340035b0 con 0x7ff148105b00 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.380+0000 7ff14cd87700 1 -- 172.21.15.70:0/1129456456 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14810b0d0 con 0x7ff148105b00 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.380+0000 7ff14cd87700 1 -- 172.21.15.70:0/1129456456 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7ff144001580 con 0x7ff148105b00 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.380+0000 7ff14cd87700 1 -- 172.21.15.70:0/1129456456 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7ff144001ce0 con 0x7ff148105b00 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.380+0000 7ff14cd87700 1 -- 172.21.15.70:0/1129456456 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7ff144005960 con 0x7ff148105b00 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 -- 172.21.15.70:0/1129456456 >> v1:172.21.15.70:6789/0 conn(0x7ff148105b00 legacy=0x7ff148105f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 -- 172.21.15.70:0/1129456456 shutdown_connections 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 -- 172.21.15.70:0/1129456456 >> 172.21.15.70:0/1129456456 conn(0x7ff1481010e0 msgr2=0x7ff148103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 -- 172.21.15.70:0/1129456456 shutdown_connections 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 -- 172.21.15.70:0/1129456456 wait complete. 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 Processor -- start 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 -- start start 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14ffed700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff14819f570 con 0x7ff148105b00 2024-02-10T07:32:30.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.381+0000 7ff14dd89700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff148105b00 0x7ff14807b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60124/0 (socket says 172.21.15.70:60124) 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff14dd89700 1 -- 172.21.15.70:0/3356761246 learned_addr learned my addr 172.21.15.70:0/3356761246 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628610483 0 0) 0x7ff14819f570 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1340054f0 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2869153182 0 0) 0x7ff1340054f0 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff14819f570 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7ff144005940 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1499841850 0 0) 0x7ff14819f570 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1481a0710 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14819f760 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14819fc60 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7ff1440052d0 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7ff144006070 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7ff144006680 con 0x7ff148105b00 2024-02-10T07:32:30.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.382+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7ff144007d80 con 0x7ff148105b00 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.383+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c005150 con 0x7ff148105b00 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.385+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7ff144001870 con 0x7ff148105b00 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.453+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff12c005400 con 0x7ff148105b00 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.453+0000 7ff13effd700 1 -- 172.21.15.70:0/3356761246 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+147 (unknown 1452402520 0 1629653342) 0x7ff144001ab0 con 0x7ff148105b00 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.454+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 >> v1:172.21.15.70:6789/0 conn(0x7ff148105b00 legacy=0x7ff14807b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.454+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 shutdown_connections 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.454+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 >> 172.21.15.70:0/3356761246 conn(0x7ff1481010e0 msgr2=0x7ff1481034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.454+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 shutdown_connections 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:29.454+0000 7ff14ffed700 1 -- 172.21.15.70:0/3356761246 wait complete. 2024-02-10T07:32:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:Restarting the monitor... 2024-02-10T07:32:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:30 smithi070 systemd[1]: Stopping Ceph mon.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:32:30.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:30 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a[29157]: 2024-02-10T07:32:30.272+0000 7f053d050700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:32:30.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:30 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a[29157]: 2024-02-10T07:32:30.272+0000 7f053d050700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-10T07:32:30.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:30 smithi070 podman[29420]: 2024-02-10 07:32:30.466772036 +0000 UTC m=+0.304368240 container died dacda01d60c0438da22eb5e16ec64e57e97824aef0cb8062e767df3f0bee4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, org.label-schema.build-date=20240131, ceph=True, GIT_CLEAN=True, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-10T07:32:30.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:30 smithi070 podman[29420]: 2024-02-10 07:32:30.797220613 +0000 UTC m=+0.634816825 container cleanup dacda01d60c0438da22eb5e16ec64e57e97824aef0cb8062e767df3f0bee4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-290212a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:32:30.973 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:30 smithi070 bash[29420]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a 2024-02-10T07:32:31.223 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:30 smithi070 podman[29434]: 2024-02-10 07:32:30.97308912 +0000 UTC m=+0.505995855 container remove dacda01d60c0438da22eb5e16ec64e57e97824aef0cb8062e767df3f0bee4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=main-290212a, ceph=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=) 2024-02-10T07:32:31.223 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-10T07:32:31.223 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 systemd[1]: Stopped Ceph mon.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:32:31.223 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 systemd[1]: Starting Ceph mon.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:32:31.654 INFO:teuthology.orchestra.run.smithi070.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-10T07:32:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 podman[29529]: 2024-02-10 07:32:31.423840542 +0000 UTC m=+0.102004443 container create b699ece04241846664645dc1675f01f2d7258650257e7d96b3e587b2b5bd9002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 podman[29529]: 2024-02-10 07:32:31.338495351 +0000 UTC m=+0.016659253 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 podman[29529]: 2024-02-10 07:32:31.592507252 +0000 UTC m=+0.270671163 container init b699ece04241846664645dc1675f01f2d7258650257e7d96b3e587b2b5bd9002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, ceph=True, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 podman[29529]: 2024-02-10 07:32:31.595253668 +0000 UTC m=+0.273417575 container start b699ece04241846664645dc1675f01f2d7258650257e7d96b3e587b2b5bd9002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 2 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: pidfile_write: ignore empty --pid-file 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: load: jerasure load: lrc 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: RocksDB version: 7.9.2 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Git sha 0 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: DB SUMMARY 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: DB Session ID: M9USKY9HB46GOZVS5WGC 2024-02-10T07:32:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: CURRENT file: CURRENT 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 85548 ; 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.error_if_exists: 0 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.create_if_missing: 0 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.paranoid_checks: 1 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.env: 0x562ce961fae0 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.info_log: 0x562ceb899900 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T07:32:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.statistics: (nil) 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.use_fsync: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_log_file_size: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.allow_fallocate: 1 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.use_direct_reads: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T07:32:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.db_log_dir: 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.wal_dir: 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.write_buffer_manager: 0x562ceb86d400 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T07:32:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T07:32:31.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T07:32:31.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T07:32:31.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T07:32:31.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T07:32:31.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.unordered_write: 0 2024-02-10T07:32:31.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T07:32:31.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.row_cache: None 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.wal_filter: None 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.two_write_queues: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.wal_compression: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.atomic_flush: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T07:32:31.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.log_readahead_size: 0 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T07:32:31.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_background_jobs: 2 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_background_compactions: -1 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_subcompactions: 1 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_open_files: -1 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T07:32:31.750 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_background_flushes: -1 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Compression algorithms supported: 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kZSTD supported: 0 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kXpressCompression supported: 0 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kZlibCompression supported: 1 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kSnappyCompression supported: 1 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kLZ4Compression supported: 1 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: kBZip2Compression supported: 0 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T07:32:31.751 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.merge_operator: 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_filter: None 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562ceb899a00) 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_type: 0 2024-02-10T07:32:31.752 INFO:journalctl@ceph.mon.a.smithi070.stdout: data_block_index_type: 0 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_shortening: 1 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: checksum: 4 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: no_block_cache: 0 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache: 0x562ceb89d090 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_options: 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: capacity : 536870912 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: num_shard_bits : 4 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: strict_capacity_limit : 0 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_compressed: (nil) 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: persistent_cache: (nil) 2024-02-10T07:32:31.753 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_size: 4096 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_size_deviation: 10 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_restart_interval: 16 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_block_restart_interval: 1 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: metadata_block_size: 4096 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: partition_filters: 0 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: use_delta_encoding: 1 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: filter_policy: bloomfilter 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: whole_key_filtering: 1 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: verify_compression: 0 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: format_version: 5 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: enable_index_compression: 1 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_align: 0 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: max_auto_readahead_size: 262144 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: prepopulate_block_cache: 0 2024-02-10T07:32:31.754 INFO:journalctl@ceph.mon.a.smithi070.stdout: initial_auto_readahead_size: 8192 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression: NoCompression 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.num_levels: 7 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T07:32:31.755 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T07:32:31.756 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T07:32:31.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T07:32:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.table_properties_collectors: 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.inplace_update_support: 0 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T07:32:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.bloom_locality: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.max_successive_merges: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.ttl: 2592000 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T07:32:31.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.enable_blob_files: false 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.min_blob_size: 0 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-10T07:32:31.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a9f10891-5cad-46bd-8776-56aef4492bd3 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550351626268, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550351627158, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 264, "table_properties": {"data_size": 80081, "index_size": 211, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10703, "raw_average_key_size": 47, "raw_value_size": 74056, "raw_average_value_size": 332, "num_data_blocks": 10, "num_entries": 223, "num_filter_entries": 223, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707550351, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a9f10891-5cad-46bd-8776-56aef4492bd3", "db_session_id": "M9USKY9HB46GOZVS5WGC", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550351627237, "job": 1, "event": "recovery_finished"} 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562ceb98e000 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: DB pointer 0x562ceb978000 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** DB Stats ** 2024-02-10T07:32:31.762 INFO:journalctl@ceph.mon.a.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** Compaction Stats [default] ** 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: L0 2/0 81.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:32:31.763 INFO:journalctl@ceph.mon.a.smithi070.stdout: Sum 2/0 81.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 132.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** Compaction Stats [default] ** 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 132.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T07:32:31.764 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative compaction: 0.00 GB write, 27.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval compaction: 0.00 GB write, 27.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: Block cache BinnedLRUCache@0x562ceb89d090#2 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-10T07:32:31.765 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: starting mon.a rank 0 at public addrs v1:172.21.15.70:6789/0 at bind addrs v1:172.21.15.70:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???) e1 preinit fsid 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???).mds e1 new map 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???).mds e1 print_map 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout: e1 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout: legacy client fscid: -1 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout: No filesystems configured 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:32:31.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:32:31.767 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:32:31.767 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 ceph-mon[29548]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-10T07:32:31.767 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 bash[29529]: b699ece04241846664645dc1675f01f2d7258650257e7d96b3e587b2b5bd9002 2024-02-10T07:32:31.767 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:31 smithi070 systemd[1]: Started Ceph mon.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:32:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:32 smithi070 ceph-mon[29548]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T07:32:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:32 smithi070 ceph-mon[29548]: monmap e1: 1 mons at {a=v1:172.21.15.70:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:32:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:32 smithi070 ceph-mon[29548]: fsmap 2024-02-10T07:32:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:32 smithi070 ceph-mon[29548]: osdmap e1: 0 total, 0 up, 0 in 2024-02-10T07:32:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:32 smithi070 ceph-mon[29548]: mgrmap e1: no daemons active 2024-02-10T07:32:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4167844115' entity='client.admin' 2024-02-10T07:32:32.751 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.089+0000 7fed68579700 1 Processor -- start 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.089+0000 7fed68579700 1 -- start start 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.089+0000 7fed68579700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed60079720 con 0x7fed601065c0 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.089+0000 7fed66315700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fed601065c0 0x7fed601089b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60140/0 (socket says 172.21.15.70:60140) 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.089+0000 7fed66315700 1 -- 172.21.15.70:0/3047730338 learned_addr learned my addr 172.21.15.70:0/3047730338 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.090+0000 7fed65313700 1 -- 172.21.15.70:0/3047730338 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991467697 0 0) 0x7fed60079720 con 0x7fed601065c0 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.090+0000 7fed65313700 1 -- 172.21.15.70:0/3047730338 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4c0035b0 con 0x7fed601065c0 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.090+0000 7fed65313700 1 -- 172.21.15.70:0/3047730338 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4078054335 0 0) 0x7fed4c0035b0 con 0x7fed601065c0 2024-02-10T07:32:32.755 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.090+0000 7fed65313700 1 -- 172.21.15.70:0/3047730338 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6010b0e0 con 0x7fed601065c0 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.090+0000 7fed65313700 1 -- 172.21.15.70:0/3047730338 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fed5c001580 con 0x7fed601065c0 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.090+0000 7fed65313700 1 -- 172.21.15.70:0/3047730338 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fed5c001ce0 con 0x7fed601065c0 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.090+0000 7fed65313700 1 -- 172.21.15.70:0/3047730338 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fed5c005960 con 0x7fed601065c0 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.091+0000 7fed68579700 1 -- 172.21.15.70:0/3047730338 >> v1:172.21.15.70:6789/0 conn(0x7fed601065c0 legacy=0x7fed601089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.091+0000 7fed68579700 1 -- 172.21.15.70:0/3047730338 shutdown_connections 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.091+0000 7fed68579700 1 -- 172.21.15.70:0/3047730338 >> 172.21.15.70:0/3047730338 conn(0x7fed600fffd0 msgr2=0x7fed601023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.091+0000 7fed68579700 1 -- 172.21.15.70:0/3047730338 shutdown_connections 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.091+0000 7fed68579700 1 -- 172.21.15.70:0/3047730338 wait complete. 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.091+0000 7fed68579700 1 Processor -- start 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.091+0000 7fed68579700 1 -- start start 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed68579700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed60199cb0 con 0x7fed601065c0 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed66315700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fed601065c0 0x7fed601995a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60152/0 (socket says 172.21.15.70:60152) 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed66315700 1 -- 172.21.15.70:0/4167844115 learned_addr learned my addr 172.21.15.70:0/4167844115 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:32.756 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792529209 0 0) 0x7fed60199cb0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4c0054f0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2834445529 0 0) 0x7fed4c0054f0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed60199cb0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fed5c001080 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 404247120 0 0) 0x7fed60199cb0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60199e40 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6019a0f0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6019dc40 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.092+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fed5c005340 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.093+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fed5c0060a0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.093+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fed5c0066b0 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.093+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7fed5c007d80 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.093+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed44005120 con 0x7fed601065c0 2024-02-10T07:32:32.757 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.095+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fed5c004ec0 con 0x7fed601065c0 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.162+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 --> v1:172.21.15.70:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fed44005950 con 0x7fed601065c0 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.164+0000 7fed577fe700 1 -- 172.21.15.70:0/4167844115 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (unknown 534127980 0 0) 0x7fed5c004a40 con 0x7fed601065c0 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.165+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 >> v1:172.21.15.70:6789/0 conn(0x7fed601065c0 legacy=0x7fed601995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.165+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 shutdown_connections 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.165+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 >> 172.21.15.70:0/4167844115 conn(0x7fed600fffd0 msgr2=0x7fed60190360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.165+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 shutdown_connections 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:32.165+0000 7fed68579700 1 -- 172.21.15.70:0/4167844115 wait complete. 2024-02-10T07:32:32.758 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-10T07:32:32.773 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:32:32.773 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mgr... 2024-02-10T07:32:32.774 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-10T07:32:32.774 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-10T07:32:33.015 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.y 2024-02-10T07:32:33.015 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.y.service: Unit ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.y.service not loaded. 2024-02-10T07:32:33.160 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-70142e68-c7e6-11ee-95b8-87774f69a715.target.wants/ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-70142e68-c7e6-11ee-95b8-87774f69a715@.service. 2024-02-10T07:32:33.376 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:33 smithi070 systemd[1]: Starting Ceph mgr.y for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:32:33.703 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:33 smithi070 podman[29769]: 2024-02-10 07:32:33.376138484 +0000 UTC m=+0.098431664 container create 2a1360cd8c565f58695d77a7743ce6f36c949b65d577b24eaa2b66c2c0f9908f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.license=GPLv2) 2024-02-10T07:32:33.703 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:33 smithi070 podman[29769]: 2024-02-10 07:32:33.295787181 +0000 UTC m=+0.018080359 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:32:33.704 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:33 smithi070 podman[29769]: 2024-02-10 07:32:33.595447294 +0000 UTC m=+0.317740470 container init 2a1360cd8c565f58695d77a7743ce6f36c949b65d577b24eaa2b66c2c0f9908f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:32:33.704 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:33 smithi070 podman[29769]: 2024-02-10 07:32:33.597856225 +0000 UTC m=+0.320149403 container start 2a1360cd8c565f58695d77a7743ce6f36c949b65d577b24eaa2b66c2c0f9908f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y, RELEASE=main-290212a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-10T07:32:33.704 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:33 smithi070 bash[29769]: 2a1360cd8c565f58695d77a7743ce6f36c949b65d577b24eaa2b66c2c0f9908f 2024-02-10T07:32:33.716 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:32:33.716 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-02-10T07:32:33.723 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:32:33.723 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-02-10T07:32:33.723 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-02-10T07:32:33.723 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T07:32:33.731 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:32:33.731 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-02-10T07:32:33.739 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:32:33.739 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-02-10T07:32:33.739 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-02-10T07:32:33.739 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-10T07:32:33.739 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr to start... 2024-02-10T07:32:33.739 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr... 2024-02-10T07:32:33.991 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:33 smithi070 systemd[1]: Started Ceph mgr.y for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:32:34.247 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:34.148+0000 7f7ccafb8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:32:34.247 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:34.246+0000 7f7ccafb8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:32:34.567 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/663298459' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:32:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:32:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "70142e68-c7e6-11ee-95b8-87774f69a715", 2024-02-10T07:32:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:32:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:32:34.806 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:32:34.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:34.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:32:34.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:34.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:32:34.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:32:34.808 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:32:34.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:32:34.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:32:34.815 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:32:34.816 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:32:34.817 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:32:26.907374+0000", 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.189+0000 7f7213bd8700 1 Processor -- start 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.189+0000 7f7213bd8700 1 -- start start 2024-02-10T07:32:34.818 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.189+0000 7f7213bd8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f720c078e90 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.190+0000 7f7211974700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f720c106560 0x7f720c108950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60182/0 (socket says 172.21.15.70:60182) 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.190+0000 7f7211974700 1 -- 172.21.15.70:0/1052592182 learned_addr learned my addr 172.21.15.70:0/1052592182 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.190+0000 7f7210972700 1 -- 172.21.15.70:0/1052592182 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90769426 0 0) 0x7f720c078e90 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.190+0000 7f7210972700 1 -- 172.21.15.70:0/1052592182 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f80035b0 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.190+0000 7f7210972700 1 -- 172.21.15.70:0/1052592182 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2649549073 0 0) 0x7f71f80035b0 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.190+0000 7f7210972700 1 -- 172.21.15.70:0/1052592182 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c10b080 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.190+0000 7f7210972700 1 -- 172.21.15.70:0/1052592182 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f7208001580 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.191+0000 7f7210972700 1 -- 172.21.15.70:0/1052592182 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f7208001c10 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.191+0000 7f7210972700 1 -- 172.21.15.70:0/1052592182 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f72080057b0 con 0x7f720c106560 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.191+0000 7f7213bd8700 1 -- 172.21.15.70:0/1052592182 >> v1:172.21.15.70:6789/0 conn(0x7f720c106560 legacy=0x7f720c108950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:34.819 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.191+0000 7f7213bd8700 1 -- 172.21.15.70:0/1052592182 shutdown_connections 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.191+0000 7f7213bd8700 1 -- 172.21.15.70:0/1052592182 >> 172.21.15.70:0/1052592182 conn(0x7f720c0fff70 msgr2=0x7f720c102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.191+0000 7f7213bd8700 1 -- 172.21.15.70:0/1052592182 shutdown_connections 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.191+0000 7f7213bd8700 1 -- 172.21.15.70:0/1052592182 wait complete. 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.192+0000 7f7213bd8700 1 Processor -- start 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.192+0000 7f7213bd8700 1 -- start start 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.192+0000 7f7213bd8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f720c199c50 con 0x7f720c106560 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7211974700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f720c106560 0x7f720c199540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60186/0 (socket says 172.21.15.70:60186) 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7211974700 1 -- 172.21.15.70:0/663298459 learned_addr learned my addr 172.21.15.70:0/663298459 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195806215 0 0) 0x7f720c199c50 con 0x7f720c106560 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f8005540 con 0x7f720c106560 2024-02-10T07:32:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 308104433 0 0) 0x7f71f8005540 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f720c199c50 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f7208001080 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2471882194 0 0) 0x7f720c199c50 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c199de0 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c19a090 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c19dbe0 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f7208004990 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.193+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f7208006150 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.194+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f7208006740 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.194+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f7208007e10 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.194+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f0005120 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.196+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f7208004eb0 con 0x7f720c106560 2024-02-10T07:32:34.821 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.268+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f71f00053d0 con 0x7f720c106560 2024-02-10T07:32:34.822 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.269+0000 7f7202ffd700 1 -- 172.21.15.70:0/663298459 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 3614698317) 0x7f7208005b20 con 0x7f720c106560 2024-02-10T07:32:34.822 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.270+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 >> v1:172.21.15.70:6789/0 conn(0x7f720c106560 legacy=0x7f720c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:34.822 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.270+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 shutdown_connections 2024-02-10T07:32:34.822 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.270+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 >> 172.21.15.70:0/663298459 conn(0x7f720c0fff70 msgr2=0x7f720c190300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:34.822 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.270+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 shutdown_connections 2024-02-10T07:32:34.822 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:34.270+0000 7f7213bd8700 1 -- 172.21.15.70:0/663298459 wait complete. 2024-02-10T07:32:34.822 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (1/15)... 2024-02-10T07:32:35.229 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:34.903+0000 7f7ccafb8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:32:35.229 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:34.995+0000 7f7ccafb8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:32:35.229 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:35.134+0000 7f7ccafb8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:32:35.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:35.228+0000 7f7ccafb8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:32:35.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:35.311+0000 7f7ccafb8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:32:35.855 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:35.569+0000 7f7ccafb8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:32:35.856 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:35.677+0000 7f7ccafb8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:32:35.856 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:35.761+0000 7f7ccafb8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:32:35.856 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:35.855+0000 7f7ccafb8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:32:36.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:36 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:36.029+0000 7f7ccafb8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:32:37.220 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:36 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:36.992+0000 7f7ccafb8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:32:37.220 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:37.125+0000 7f7ccafb8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:32:37.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:37.466+0000 7f7ccafb8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:32:37.840 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3999671135' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:32:37.840 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:37.560+0000 7f7ccafb8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:32:37.840 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:37.753+0000 7f7ccafb8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:32:37.844 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "70142e68-c7e6-11ee-95b8-87774f69a715", 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:32:37.845 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:32:37.846 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:32:37.853 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:37.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:32:37.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:32:26.907374+0000", 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.226+0000 7f3582bf6700 1 Processor -- start 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.226+0000 7f3582bf6700 1 -- start start 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.226+0000 7f3582bf6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f357c109f40 con 0x7f357c105b10 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.226+0000 7f3580992700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f357c105b10 0x7f357c105f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35734/0 (socket says 172.21.15.70:35734) 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.226+0000 7f3580992700 1 -- 172.21.15.70:0/1469879713 learned_addr learned my addr 172.21.15.70:0/1469879713 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.227+0000 7f357b7fe700 1 -- 172.21.15.70:0/1469879713 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824004148 0 0) 0x7f357c109f40 con 0x7f357c105b10 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.227+0000 7f357b7fe700 1 -- 172.21.15.70:0/1469879713 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35640035b0 con 0x7f357c105b10 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.227+0000 7f357b7fe700 1 -- 172.21.15.70:0/1469879713 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2477828666 0 0) 0x7f35640035b0 con 0x7f357c105b10 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.227+0000 7f357b7fe700 1 -- 172.21.15.70:0/1469879713 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c10b0e0 con 0x7f357c105b10 2024-02-10T07:32:37.856 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.227+0000 7f357b7fe700 1 -- 172.21.15.70:0/1469879713 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3574001900 con 0x7f357c105b10 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.227+0000 7f357b7fe700 1 -- 172.21.15.70:0/1469879713 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f3574002060 con 0x7f357c105b10 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.227+0000 7f357b7fe700 1 -- 172.21.15.70:0/1469879713 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3574005ce0 con 0x7f357c105b10 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.228+0000 7f3582bf6700 1 -- 172.21.15.70:0/1469879713 >> v1:172.21.15.70:6789/0 conn(0x7f357c105b10 legacy=0x7f357c105f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.228+0000 7f3582bf6700 1 -- 172.21.15.70:0/1469879713 shutdown_connections 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.228+0000 7f3582bf6700 1 -- 172.21.15.70:0/1469879713 >> 172.21.15.70:0/1469879713 conn(0x7f357c1010d0 msgr2=0x7f357c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.228+0000 7f3582bf6700 1 -- 172.21.15.70:0/1469879713 shutdown_connections 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.228+0000 7f3582bf6700 1 -- 172.21.15.70:0/1469879713 wait complete. 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.228+0000 7f3582bf6700 1 Processor -- start 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.228+0000 7f3582bf6700 1 -- start start 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3582bf6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f357c199ce0 con 0x7f357c105b10 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3580992700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f357c105b10 0x7f357c1995d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35738/0 (socket says 172.21.15.70:35738) 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3580992700 1 -- 172.21.15.70:0/3999671135 learned_addr learned my addr 172.21.15.70:0/3999671135 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276372273 0 0) 0x7f357c199ce0 con 0x7f357c105b10 2024-02-10T07:32:37.857 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35640054f0 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3241365984 0 0) 0x7f35640054f0 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f357c199ce0 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3574005cc0 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 220590359 0 0) 0x7f357c199ce0 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c199e70 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357c19a120 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.229+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f3574005650 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.230+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3574006570 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.230+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f3574006b80 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.230+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c19dc20 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.230+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f35740073f0 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.230+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357c193230 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.233+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f3574001bf0 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.305+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f357c02d810 con 0x7f357c105b10 2024-02-10T07:32:37.858 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.305+0000 7f3579ffb700 1 -- 172.21.15.70:0/3999671135 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 3102941683) 0x7f3574001e10 con 0x7f357c105b10 2024-02-10T07:32:37.859 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.307+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 >> v1:172.21.15.70:6789/0 conn(0x7f357c105b10 legacy=0x7f357c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:37.859 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.307+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 shutdown_connections 2024-02-10T07:32:37.859 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.307+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 >> 172.21.15.70:0/3999671135 conn(0x7f357c1010d0 msgr2=0x7f357c190370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:37.859 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.307+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 shutdown_connections 2024-02-10T07:32:37.859 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:37.307+0000 7f3582bf6700 1 -- 172.21.15.70:0/3999671135 wait complete. 2024-02-10T07:32:37.859 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (2/15)... 2024-02-10T07:32:38.219 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:37.939+0000 7f7ccafb8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:32:38.481 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:38 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:38.218+0000 7f7ccafb8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:32:38.481 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:38 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:38.480+0000 7f7ccafb8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:32:38.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:38 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:38.713+0000 7f7ccafb8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:32:39.969 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:39 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:39.497+0000 7f7ccafb8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:32:40.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:40.063+0000 7f7ccafb8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:32:40.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:40.159+0000 7f7ccafb8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:32:40.563 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:40.375+0000 7f7ccafb8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:32:40.563 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: Activating manager daemon y 2024-02-10T07:32:40.563 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: mgrmap e2: y(active, starting, since 0.00306481s) 2024-02-10T07:32:40.563 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: Manager daemon y is now available 2024-02-10T07:32:40.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:32:40.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' 2024-02-10T07:32:40.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:32:40.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' 2024-02-10T07:32:40.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:40 smithi070 ceph-mon[29548]: from='mgr.14100 v1:172.21.15.70:0/2224872231' entity='mgr.y' 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "70142e68-c7e6-11ee-95b8-87774f69a715", 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:41.075 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:32:41.076 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:32:41.077 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:32:41.078 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:32:26.907374+0000", 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.486+0000 7f3f3fbeb700 1 Processor -- start 2024-02-10T07:32:41.079 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.486+0000 7f3f3fbeb700 1 -- start start 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.486+0000 7f3f3fbeb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3f38109f30 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.486+0000 7f3f3d987700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3f38105b00 0x7f3f38105f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35796/0 (socket says 172.21.15.70:35796) 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.486+0000 7f3f3d987700 1 -- 172.21.15.70:0/3578827229 learned_addr learned my addr 172.21.15.70:0/3578827229 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3c985700 1 -- 172.21.15.70:0/3578827229 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386840335 0 0) 0x7f3f38109f30 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3c985700 1 -- 172.21.15.70:0/3578827229 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f240035b0 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3c985700 1 -- 172.21.15.70:0/3578827229 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1301905859 0 0) 0x7f3f240035b0 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3c985700 1 -- 172.21.15.70:0/3578827229 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3810b0d0 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3c985700 1 -- 172.21.15.70:0/3578827229 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3f34001580 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3c985700 1 -- 172.21.15.70:0/3578827229 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f3f34001ce0 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3c985700 1 -- 172.21.15.70:0/3578827229 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3f34005960 con 0x7f3f38105b00 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.487+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/3578827229 >> v1:172.21.15.70:6789/0 conn(0x7f3f38105b00 legacy=0x7f3f38105f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/3578827229 shutdown_connections 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/3578827229 >> 172.21.15.70:0/3578827229 conn(0x7f3f381010e0 msgr2=0x7f3f38103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:41.080 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/3578827229 shutdown_connections 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/3578827229 wait complete. 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3fbeb700 1 Processor -- start 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3fbeb700 1 -- start start 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3fbeb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3f3819f570 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3d987700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3f38105b00 0x7f3f3807b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35812/0 (socket says 172.21.15.70:35812) 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.488+0000 7f3f3d987700 1 -- 172.21.15.70:0/2490363151 learned_addr learned my addr 172.21.15.70:0/2490363151 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976359384 0 0) 0x7f3f3819f570 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f240054f0 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3200258786 0 0) 0x7f3f240054f0 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f3819f570 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3f34005940 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1000679580 0 0) 0x7f3f3819f570 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f381a0710 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3819f760 con 0x7f3f38105b00 2024-02-10T07:32:41.081 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3819fc60 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f3f340052d0 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.489+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3f340061c0 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.490+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 2) v1 ==== 45688+0+0 (unknown 723840842 0 0) 0x7f3f340115a0 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.490+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c005150 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.490+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f3f34046d20 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.492+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f3f34016750 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.561+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3f1c005400 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.562+0000 7f3f2effd700 1 -- 172.21.15.70:0/2490363151 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 910015026) 0x7f3f34014d50 con 0x7f3f38105b00 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.563+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 >> v1:172.21.15.70:6789/0 conn(0x7f3f38105b00 legacy=0x7f3f3807b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.563+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 shutdown_connections 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.563+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 >> 172.21.15.70:0/2490363151 conn(0x7f3f381010e0 msgr2=0x7f3f381034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.563+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 shutdown_connections 2024-02-10T07:32:41.082 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:40.563+0000 7f3f3fbeb700 1 -- 172.21.15.70:0/2490363151 wait complete. 2024-02-10T07:32:41.083 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (3/15)... 2024-02-10T07:32:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2490363151' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:32:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:41 smithi070 ceph-mon[29548]: mgrmap e3: y(active, since 1.00617s) 2024-02-10T07:32:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:43 smithi070 ceph-mon[29548]: mgrmap e4: y(active, since 2s) 2024-02-10T07:32:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3306911053' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:32:52.123 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:52.123 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "70142e68-c7e6-11ee-95b8-87774f69a715", 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:52.124 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:32:52.125 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:52.126 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:32:26.907374+0000", 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:32:52.127 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.466+0000 7f9893a7f700 1 Processor -- start 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.466+0000 7f9893a7f700 1 -- start start 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.466+0000 7f9893a7f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f988c108e40 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.466+0000 7f989181b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f988c104a10 0x7f988c104e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51514/0 (socket says 172.21.15.70:51514) 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.466+0000 7f989181b700 1 -- 172.21.15.70:0/826819977 learned_addr learned my addr 172.21.15.70:0/826819977 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.466+0000 7f9890819700 1 -- 172.21.15.70:0/826819977 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499746020 0 0) 0x7f988c108e40 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.466+0000 7f9890819700 1 -- 172.21.15.70:0/826819977 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98780035b0 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9890819700 1 -- 172.21.15.70:0/826819977 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2087871159 0 0) 0x7f98780035b0 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9890819700 1 -- 172.21.15.70:0/826819977 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c109fe0 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9890819700 1 -- 172.21.15.70:0/826819977 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9888001580 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9890819700 1 -- 172.21.15.70:0/826819977 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f9888001c10 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9890819700 1 -- 172.21.15.70:0/826819977 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9888005780 con 0x7f988c104a10 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9893a7f700 1 -- 172.21.15.70:0/826819977 >> v1:172.21.15.70:6789/0 conn(0x7f988c104a10 legacy=0x7f988c104e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:52.128 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9893a7f700 1 -- 172.21.15.70:0/826819977 shutdown_connections 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9893a7f700 1 -- 172.21.15.70:0/826819977 >> 172.21.15.70:0/826819977 conn(0x7f988c0fffd0 msgr2=0x7f988c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.467+0000 7f9893a7f700 1 -- 172.21.15.70:0/826819977 shutdown_connections 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.468+0000 7f9893a7f700 1 -- 172.21.15.70:0/826819977 wait complete. 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.468+0000 7f9893a7f700 1 Processor -- start 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.468+0000 7f9893a7f700 1 -- start start 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.468+0000 7f9893a7f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f988c199cb0 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.468+0000 7f989181b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f988c104a10 0x7f988c1995a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51530/0 (socket says 172.21.15.70:51530) 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.468+0000 7f989181b700 1 -- 172.21.15.70:0/3306911053 learned_addr learned my addr 172.21.15.70:0/3306911053 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336160892 0 0) 0x7f988c199cb0 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9878005540 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 999425683 0 0) 0x7f9878005540 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f988c199cb0 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9888001080 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3372155384 0 0) 0x7f988c199cb0 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c199e40 con 0x7f988c104a10 2024-02-10T07:32:52.129 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c19a0f0 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c19dc40 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f9888004870 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.469+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9888006070 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.470+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 4) v1 ==== 45886+0+0 (unknown 2140634397 0 0) 0x7f9888011600 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.470+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9870005150 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.470+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f98880472c0 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.475+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f9888012e20 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.752+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9870005400 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.752+0000 7f9882ffd700 1 -- 172.21.15.70:0/3306911053 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 4060034746) 0x7f9888014f00 con 0x7f988c104a10 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.754+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 >> v1:172.21.15.70:6800/1444653888 conn(0x7f987803a070 legacy=0x7f987803c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.754+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 >> v1:172.21.15.70:6789/0 conn(0x7f988c104a10 legacy=0x7f988c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.754+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 shutdown_connections 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.754+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 >> 172.21.15.70:0/3306911053 conn(0x7f988c0fffd0 msgr2=0x7f988c078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:52.130 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.754+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 shutdown_connections 2024-02-10T07:32:52.131 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:46.754+0000 7f9893a7f700 1 -- 172.21.15.70:0/3306911053 wait complete. 2024-02-10T07:32:52.131 INFO:teuthology.orchestra.run.smithi070.stdout:mgr is available 2024-02-10T07:32:53.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2515722643' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T07:32:53.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T07:32:53.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.587+0000 7f605e636700 1 Processor -- start 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.587+0000 7f605e636700 1 -- start start 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.587+0000 7f605e636700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6058108e40 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6057fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6058104a10 0x7f6058104e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51536/0 (socket says 172.21.15.70:51536) 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6057fff700 1 -- 172.21.15.70:0/2950598137 learned_addr learned my addr 172.21.15.70:0/2950598137 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6056ffd700 1 -- 172.21.15.70:0/2950598137 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419060640 0 0) 0x7f6058108e40 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6056ffd700 1 -- 172.21.15.70:0/2950598137 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60400035b0 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6056ffd700 1 -- 172.21.15.70:0/2950598137 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497696336 0 0) 0x7f60400035b0 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6056ffd700 1 -- 172.21.15.70:0/2950598137 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058109fe0 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6056ffd700 1 -- 172.21.15.70:0/2950598137 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f6048001580 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6056ffd700 1 -- 172.21.15.70:0/2950598137 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f6048001c10 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.588+0000 7f6056ffd700 1 -- 172.21.15.70:0/2950598137 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f6048005780 con 0x7f6058104a10 2024-02-10T07:32:53.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.589+0000 7f605e636700 1 -- 172.21.15.70:0/2950598137 >> v1:172.21.15.70:6789/0 conn(0x7f6058104a10 legacy=0x7f6058104e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.589+0000 7f605e636700 1 -- 172.21.15.70:0/2950598137 shutdown_connections 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.589+0000 7f605e636700 1 -- 172.21.15.70:0/2950598137 >> 172.21.15.70:0/2950598137 conn(0x7f60580fffd0 msgr2=0x7f60581023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.589+0000 7f605e636700 1 -- 172.21.15.70:0/2950598137 shutdown_connections 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.589+0000 7f605e636700 1 -- 172.21.15.70:0/2950598137 wait complete. 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.590+0000 7f605e636700 1 Processor -- start 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.590+0000 7f605e636700 1 -- start start 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.590+0000 7f605e636700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6058198c10 con 0x7f6058104a10 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.590+0000 7f6057fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6058104a10 0x7f6058198500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51538/0 (socket says 172.21.15.70:51538) 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.590+0000 7f6057fff700 1 -- 172.21.15.70:0/2515722643 learned_addr learned my addr 172.21.15.70:0/2515722643 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.590+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133396139 0 0) 0x7f6058198c10 con 0x7f6058104a10 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.590+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6040005540 con 0x7f6058104a10 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1886970555 0 0) 0x7f6040005540 con 0x7f6058104a10 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6058198c10 con 0x7f6058104a10 2024-02-10T07:32:53.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f6048001080 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2425102349 0 0) 0x7f6058198c10 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058198da0 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6058199050 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f605819cb50 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f6048001a50 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.591+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f6048006150 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.592+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 4) v1 ==== 45886+0+0 (unknown 2140634397 0 0) 0x7f6048011620 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.592+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f60480480a0 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.592+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6044005150 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.597+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6048012e30 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.789+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6044005980 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.789+0000 7f60557fa700 1 -- 172.21.15.70:0/2515722643 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (unknown 1187553405 0 2057907182) 0x7f6048014f10 con 0x7f6058104a10 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.792+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 >> v1:172.21.15.70:6800/1444653888 conn(0x7f604003a070 legacy=0x7f604003c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.792+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 >> v1:172.21.15.70:6789/0 conn(0x7f6058104a10 legacy=0x7f6058198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:53.304 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.792+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 shutdown_connections 2024-02-10T07:32:53.304 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.792+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 >> 172.21.15.70:0/2515722643 conn(0x7f60580fffd0 msgr2=0x7f605818f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:53.304 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.792+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 shutdown_connections 2024-02-10T07:32:53.304 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:52.792+0000 7f605e636700 1 -- 172.21.15.70:0/2515722643 wait complete. 2024-02-10T07:32:53.304 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling cephadm module... 2024-02-10T07:32:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1553415028' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-10T07:32:54.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:54 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ignoring --setuser ceph since I am not root 2024-02-10T07:32:54.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:54 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ignoring --setgroup ceph since I am not root 2024-02-10T07:32:54.955 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:54 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:54.552+0000 7f4962b7f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:32:54.956 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:54 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:54.645+0000 7f4962b7f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:32:55.239 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1553415028' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-10T07:32:55.239 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-mon[29548]: mgrmap e5: y(active, since 13s) 2024-02-10T07:32:55.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.694+0000 7f9ee025a700 1 Processor -- start 2024-02-10T07:32:55.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.694+0000 7f9ee025a700 1 -- start start 2024-02-10T07:32:55.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.694+0000 7f9ee025a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9ed807a330 con 0x7f9ed807b730 2024-02-10T07:32:55.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.694+0000 7f9eddff6700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9ed807b730 0x7f9ed8079b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51540/0 (socket says 172.21.15.70:51540) 2024-02-10T07:32:55.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.694+0000 7f9eddff6700 1 -- 172.21.15.70:0/3783100334 learned_addr learned my addr 172.21.15.70:0/3783100334 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:55.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9edcff4700 1 -- 172.21.15.70:0/3783100334 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840687159 0 0) 0x7f9ed807a330 con 0x7f9ed807b730 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9edcff4700 1 -- 172.21.15.70:0/3783100334 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec40035b0 con 0x7f9ed807b730 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9edcff4700 1 -- 172.21.15.70:0/3783100334 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 268927828 0 0) 0x7f9ec40035b0 con 0x7f9ed807b730 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9edcff4700 1 -- 172.21.15.70:0/3783100334 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed807a4c0 con 0x7f9ed807b730 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9edcff4700 1 -- 172.21.15.70:0/3783100334 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9ed4001580 con 0x7f9ed807b730 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9edcff4700 1 -- 172.21.15.70:0/3783100334 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f9ed4001ce0 con 0x7f9ed807b730 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9edcff4700 1 -- 172.21.15.70:0/3783100334 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9ed4005960 con 0x7f9ed807b730 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9ee025a700 1 -- 172.21.15.70:0/3783100334 >> v1:172.21.15.70:6789/0 conn(0x7f9ed807b730 legacy=0x7f9ed8079b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.695+0000 7f9ee025a700 1 -- 172.21.15.70:0/3783100334 shutdown_connections 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9ee025a700 1 -- 172.21.15.70:0/3783100334 >> 172.21.15.70:0/3783100334 conn(0x7f9ed81010e0 msgr2=0x7f9ed8103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9ee025a700 1 -- 172.21.15.70:0/3783100334 shutdown_connections 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9ee025a700 1 -- 172.21.15.70:0/3783100334 wait complete. 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9ee025a700 1 Processor -- start 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9ee025a700 1 -- start start 2024-02-10T07:32:55.247 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9ee025a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9ed819be60 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9eddff6700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9ed807b730 0x7f9ed819b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51548/0 (socket says 172.21.15.70:51548) 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.696+0000 7f9eddff6700 1 -- 172.21.15.70:0/1553415028 learned_addr learned my addr 172.21.15.70:0/1553415028 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299073205 0 0) 0x7f9ed819be60 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec40054f0 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1044818172 0 0) 0x7f9ec40054f0 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ed819be60 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9ed4001080 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 858904474 0 0) 0x7f9ed819be60 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed819bff0 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed819c2a0 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed819fdf0 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f9ed4004a40 con 0x7f9ed807b730 2024-02-10T07:32:55.248 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.697+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9ed4006120 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.698+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 4) v1 ==== 45886+0+0 (unknown 2140634397 0 0) 0x7f9ed40115d0 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.698+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc005150 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.698+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f9ed4047170 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.703+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f9ed4012de0 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:53.949+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f9ebc005400 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.000+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (unknown 2263024820 0 0) 0x7f9ed4014ec0 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.002+0000 7f9eceffd700 1 -- 172.21.15.70:0/1553415028 <== mon.0 v1:172.21.15.70:6789/0 11 ==== mgrmap(e 5) v1 ==== 45897+0+0 (unknown 1812618396 0 0) 0x7f9ed4045bc0 con 0x7f9ed807b730 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.003+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 >> v1:172.21.15.70:6800/1444653888 conn(0x7f9ec403a0d0 legacy=0x7f9ec403c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.003+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 >> v1:172.21.15.70:6789/0 conn(0x7f9ed807b730 legacy=0x7f9ed819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.003+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 shutdown_connections 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.003+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 >> 172.21.15.70:0/1553415028 conn(0x7f9ed81010e0 msgr2=0x7f9ed8109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.003+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 shutdown_connections 2024-02-10T07:32:55.249 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:54.003+0000 7f9ee025a700 1 -- 172.21.15.70:0/1553415028 wait complete. 2024-02-10T07:32:55.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:55.285+0000 7f4962b7f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:32:55.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:55.377+0000 7f4962b7f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:32:55.880 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:55.521+0000 7f4962b7f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:32:55.880 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:55.617+0000 7f4962b7f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:32:55.881 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:55.706+0000 7f4962b7f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:32:56.169 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:32:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3557829308' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T07:32:56.169 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:55 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:55.974+0000 7f4962b7f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:32:56.169 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:56 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:56.083+0000 7f4962b7f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:32:56.169 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:56 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:56.168+0000 7f4962b7f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:32:56.441 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:56 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:56.265+0000 7f4962b7f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:32:56.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:32:56.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-10T07:32:56.545 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.674+0000 7f969141b700 1 Processor -- start 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.674+0000 7f969141b700 1 -- start start 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.674+0000 7f969141b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f968c079680 con 0x7f968c106650 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.674+0000 7f968affd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f968c106650 0x7f968c108a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51570/0 (socket says 172.21.15.70:51570) 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.674+0000 7f968affd700 1 -- 172.21.15.70:0/1425613892 learned_addr learned my addr 172.21.15.70:0/1425613892 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.674+0000 7f9689ffb700 1 -- 172.21.15.70:0/1425613892 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181327367 0 0) 0x7f968c079680 con 0x7f968c106650 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.674+0000 7f9689ffb700 1 -- 172.21.15.70:0/1425613892 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96740035b0 con 0x7f968c106650 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.675+0000 7f9689ffb700 1 -- 172.21.15.70:0/1425613892 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955779378 0 0) 0x7f96740035b0 con 0x7f968c106650 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.675+0000 7f9689ffb700 1 -- 172.21.15.70:0/1425613892 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c10b170 con 0x7f968c106650 2024-02-10T07:32:56.546 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.675+0000 7f9689ffb700 1 -- 172.21.15.70:0/1425613892 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f967c001580 con 0x7f968c106650 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.675+0000 7f9689ffb700 1 -- 172.21.15.70:0/1425613892 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f967c001c10 con 0x7f968c106650 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.675+0000 7f9689ffb700 1 -- 172.21.15.70:0/1425613892 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f967c0057b0 con 0x7f968c106650 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.676+0000 7f969141b700 1 -- 172.21.15.70:0/1425613892 >> v1:172.21.15.70:6789/0 conn(0x7f968c106650 legacy=0x7f968c108a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.676+0000 7f969141b700 1 -- 172.21.15.70:0/1425613892 shutdown_connections 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.676+0000 7f969141b700 1 -- 172.21.15.70:0/1425613892 >> 172.21.15.70:0/1425613892 conn(0x7f968c0fffe0 msgr2=0x7f968c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.676+0000 7f969141b700 1 -- 172.21.15.70:0/1425613892 shutdown_connections 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.676+0000 7f969141b700 1 -- 172.21.15.70:0/1425613892 wait complete. 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.677+0000 7f969141b700 1 Processor -- start 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.677+0000 7f969141b700 1 -- start start 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.677+0000 7f969141b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f968c10afe0 con 0x7f968c106650 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.677+0000 7f968affd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f968c106650 0x7f968c199580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51586/0 (socket says 172.21.15.70:51586) 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.677+0000 7f968affd700 1 -- 172.21.15.70:0/3557829308 learned_addr learned my addr 172.21.15.70:0/3557829308 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.677+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287954106 0 0) 0x7f968c10afe0 con 0x7f968c106650 2024-02-10T07:32:56.547 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.677+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9674005540 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 487284492 0 0) 0x7f9674005540 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f968c10afe0 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f967c001080 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1614795582 0 0) 0x7f968c10afe0 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c199c90 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c199f40 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c19a1f0 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f967c004990 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.678+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f967c006150 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.679+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9678005120 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.679+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) v1 ==== 45897+0+0 (unknown 1812618396 0 0) 0x7f967c0062b0 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.679+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f967c047180 con 0x7f968c106650 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.679+0000 7f968a7fc700 1 -- 172.21.15.70:0/3557829308 >> v1:172.21.15.70:6800/1444653888 conn(0x7f967403a110 legacy=0x7f967403c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1444653888 2024-02-10T07:32:56.548 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.686+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f967c007b10 con 0x7f968c106650 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.880+0000 7f968a7fc700 1 -- 172.21.15.70:0/3557829308 >> v1:172.21.15.70:6800/1444653888 conn(0x7f967403a110 legacy=0x7f967403c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1444653888 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.938+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9678005be0 con 0x7f968c106650 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.938+0000 7f9673fff700 1 -- 172.21.15.70:0/3557829308 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7f967c014f30 con 0x7f968c106650 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.941+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 >> v1:172.21.15.70:6800/1444653888 conn(0x7f967403a110 legacy=0x7f967403c5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.941+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 >> v1:172.21.15.70:6789/0 conn(0x7f968c106650 legacy=0x7f968c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.941+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 shutdown_connections 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.941+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 >> 172.21.15.70:0/3557829308 conn(0x7f968c0fffe0 msgr2=0x7f968c190340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.941+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 shutdown_connections 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:55.941+0000 7f969141b700 1 -- 172.21.15.70:0/3557829308 wait complete. 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-02-10T07:32:56.549 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 5... 2024-02-10T07:32:56.728 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:56 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:56.440+0000 7f4962b7f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:32:57.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:57 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:57.409+0000 7f4962b7f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:32:57.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:57 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:57.538+0000 7f4962b7f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:32:58.146 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:57 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:57.864+0000 7f4962b7f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:32:58.147 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:57 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:57.953+0000 7f4962b7f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:32:58.147 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:58 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:58.146+0000 7f4962b7f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:32:58.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:58 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:58.327+0000 7f4962b7f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:32:58.867 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:58 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:58.605+0000 7f4962b7f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:32:59.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:58 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:58.866+0000 7f4962b7f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:32:59.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:59 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:59.100+0000 7f4962b7f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:33:00.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:32:59 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:32:59.888+0000 7f4962b7f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:33:00.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:00.451+0000 7f4962b7f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:33:00.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:00.548+0000 7f4962b7f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:33:01.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:00.760+0000 7f4962b7f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:33:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: Active manager daemon y restarted 2024-02-10T07:33:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: Activating manager daemon y 2024-02-10T07:33:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: osdmap e2: 0 total, 0 up, 0 in 2024-02-10T07:33:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: mgrmap e6: y(active, starting, since 0.0032886s) 2024-02-10T07:33:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: Manager daemon y is now available 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:00 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:02.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:01 smithi070 ceph-mon[29548]: Found migration_current of "None". Setting to last migration. 2024-02-10T07:33:02.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:01 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:33:02.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:01 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:33:02.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:01 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:02.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:01 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:02.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:01 smithi070 ceph-mon[29548]: mgrmap e7: y(active, since 1.00504s) 2024-02-10T07:33:02.401 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:33:02.401 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-10T07:33:02.402 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T07:33:02.402 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:33:02.402 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.948+0000 7f58aa8d3700 1 Processor -- start 2024-02-10T07:33:02.402 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.948+0000 7f58aa8d3700 1 -- start start 2024-02-10T07:33:02.402 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58aa8d3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f58a4108e80 con 0x7f58a4105f90 2024-02-10T07:33:02.402 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a3fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f58a4105f90 0x7f58a41063b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43828/0 (socket says 172.21.15.70:43828) 2024-02-10T07:33:02.402 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a3fff700 1 -- 172.21.15.70:0/2941511911 learned_addr learned my addr 172.21.15.70:0/2941511911 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a2ffd700 1 -- 172.21.15.70:0/2941511911 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235289416 0 0) 0x7f58a4108e80 con 0x7f58a4105f90 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a2ffd700 1 -- 172.21.15.70:0/2941511911 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588c0035b0 con 0x7f58a4105f90 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a2ffd700 1 -- 172.21.15.70:0/2941511911 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2437023913 0 0) 0x7f588c0035b0 con 0x7f58a4105f90 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a2ffd700 1 -- 172.21.15.70:0/2941511911 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a410a020 con 0x7f58a4105f90 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a2ffd700 1 -- 172.21.15.70:0/2941511911 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5894001580 con 0x7f58a4105f90 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a2ffd700 1 -- 172.21.15.70:0/2941511911 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f5894001ce0 con 0x7f58a4105f90 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.949+0000 7f58a2ffd700 1 -- 172.21.15.70:0/2941511911 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5894005960 con 0x7f58a4105f90 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.950+0000 7f58aa8d3700 1 -- 172.21.15.70:0/2941511911 >> v1:172.21.15.70:6789/0 conn(0x7f58a4105f90 legacy=0x7f58a41063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.950+0000 7f58aa8d3700 1 -- 172.21.15.70:0/2941511911 shutdown_connections 2024-02-10T07:33:02.403 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.950+0000 7f58aa8d3700 1 -- 172.21.15.70:0/2941511911 >> 172.21.15.70:0/2941511911 conn(0x7f58a40ff6f0 msgr2=0x7f58a4101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.950+0000 7f58aa8d3700 1 -- 172.21.15.70:0/2941511911 shutdown_connections 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.950+0000 7f58aa8d3700 1 -- 172.21.15.70:0/2941511911 wait complete. 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.951+0000 7f58aa8d3700 1 Processor -- start 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.951+0000 7f58aa8d3700 1 -- start start 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.951+0000 7f58aa8d3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f58a419bea0 con 0x7f58a4105f90 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.951+0000 7f58a3fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f58a4105f90 0x7f58a419b790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43834/0 (socket says 172.21.15.70:43834) 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.951+0000 7f58a3fff700 1 -- 172.21.15.70:0/4118021238 learned_addr learned my addr 172.21.15.70:0/4118021238 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.951+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468095694 0 0) 0x7f58a419bea0 con 0x7f58a4105f90 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.951+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588c0054f0 con 0x7f58a4105f90 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3417679097 0 0) 0x7f588c0054f0 con 0x7f58a4105f90 2024-02-10T07:33:02.404 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f58a419bea0 con 0x7f58a4105f90 2024-02-10T07:33:02.406 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5894001080 con 0x7f58a4105f90 2024-02-10T07:33:02.406 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 588581832 0 0) 0x7f58a419bea0 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a419c030 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a419c2e0 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a419fe30 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f5894005340 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.952+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5894006190 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.953+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 5) v1 ==== 45897+0+0 (unknown 1812618396 0 0) 0x7f5894011670 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.953+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6800/1444653888 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58a4063350 con 0x7f588c03a170 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.953+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 3767258219 0 0) 0x7f5894048000 con 0x7f58a4105f90 2024-02-10T07:33:02.407 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:56.953+0000 7f58a37fe700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6800/1444653888 conn(0x7f588c03a170 legacy=0x7f588c03c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1444653888 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:57.154+0000 7f58a37fe700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6800/1444653888 conn(0x7f588c03a170 legacy=0x7f588c03c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1444653888 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:57.555+0000 7f58a37fe700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6800/1444653888 conn(0x7f588c03a170 legacy=0x7f588c03c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1444653888 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:58.356+0000 7f58a37fe700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6800/1444653888 conn(0x7f588c03a170 legacy=0x7f588c03c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1444653888 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:32:59.957+0000 7f58a37fe700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6800/1444653888 conn(0x7f588c03a170 legacy=0x7f588c03c630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1444653888 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:00.776+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mgrmap(e 6) v1 ==== 45699+0+0 (unknown 88998245 0 0) 0x7f5894012b50 con 0x7f58a4105f90 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:00.776+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6800/1444653888 conn(0x7f588c03a170 legacy=0x7f588c03c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.777+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mgrmap(e 7) v1 ==== 45791+0+0 (unknown 428442895 0 0) 0x7f5894048cb0 con 0x7f58a4105f90 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.777+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6800/1990687880 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58a4063350 con 0x7f588c03e960 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.778+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (unknown 0 0 4109177037) 0x7f58a4063350 con 0x7f588c03e960 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.789+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 --> v1:172.21.15.70:6800/1990687880 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f58a4109bf0 con 0x7f588c03e960 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.789+0000 7f58a17fa700 1 -- 172.21.15.70:0/4118021238 <== mgr.14120 v1:172.21.15.70:6800/1990687880 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (unknown 0 0 96372106) 0x7f58a4109bf0 con 0x7f588c03e960 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.789+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6800/1990687880 conn(0x7f588c03e960 legacy=0x7f588c040d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.789+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 >> v1:172.21.15.70:6789/0 conn(0x7f58a4105f90 legacy=0x7f58a419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:02.408 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.789+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 shutdown_connections 2024-02-10T07:33:02.409 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.789+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 >> 172.21.15.70:0/4118021238 conn(0x7f58a40ff6f0 msgr2=0x7f58a4101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:02.409 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.790+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 shutdown_connections 2024-02-10T07:33:02.409 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:01.790+0000 7f58aa8d3700 1 -- 172.21.15.70:0/4118021238 wait complete. 2024-02-10T07:33:02.409 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 5 is available 2024-02-10T07:33:02.409 INFO:teuthology.orchestra.run.smithi070.stdout:Setting orchestrator backend to cephadm... 2024-02-10T07:33:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:02 smithi070 ceph-mon[29548]: from='client.14124 v1:172.21.15.70:0/4118021238' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T07:33:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:02 smithi070 ceph-mon[29548]: from='client.14124 v1:172.21.15.70:0/4118021238' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T07:33:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:02 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:02] ENGINE Bus STARTING 2024-02-10T07:33:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:02 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.810+0000 7f726c1bd700 1 Processor -- start 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.810+0000 7f726c1bd700 1 -- start start 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.810+0000 7f726c1bd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7264079700 con 0x7f72641043d0 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.810+0000 7f7269f59700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72641043d0 0x7f72641047f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43888/0 (socket says 172.21.15.70:43888) 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.810+0000 7f7269f59700 1 -- 172.21.15.70:0/2216202093 learned_addr learned my addr 172.21.15.70:0/2216202093 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.811+0000 7f7268f57700 1 -- 172.21.15.70:0/2216202093 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611992850 0 0) 0x7f7264079700 con 0x7f72641043d0 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.811+0000 7f7268f57700 1 -- 172.21.15.70:0/2216202093 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72500035b0 con 0x7f72641043d0 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.811+0000 7f7268f57700 1 -- 172.21.15.70:0/2216202093 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2388822182 0 0) 0x7f72500035b0 con 0x7f72641043d0 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.811+0000 7f7268f57700 1 -- 172.21.15.70:0/2216202093 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7264105f10 con 0x7f72641043d0 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.811+0000 7f7268f57700 1 -- 172.21.15.70:0/2216202093 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f7260001900 con 0x7f72641043d0 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.811+0000 7f7268f57700 1 -- 172.21.15.70:0/2216202093 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f7260002060 con 0x7f72641043d0 2024-02-10T07:33:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.811+0000 7f7268f57700 1 -- 172.21.15.70:0/2216202093 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f7260005ce0 con 0x7f72641043d0 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.812+0000 7f726c1bd700 1 -- 172.21.15.70:0/2216202093 >> v1:172.21.15.70:6789/0 conn(0x7f72641043d0 legacy=0x7f72641047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.812+0000 7f726c1bd700 1 -- 172.21.15.70:0/2216202093 shutdown_connections 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.812+0000 7f726c1bd700 1 -- 172.21.15.70:0/2216202093 >> 172.21.15.70:0/2216202093 conn(0x7f72640fffd0 msgr2=0x7f72641023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.812+0000 7f726c1bd700 1 -- 172.21.15.70:0/2216202093 shutdown_connections 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.812+0000 7f726c1bd700 1 -- 172.21.15.70:0/2216202093 wait complete. 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.812+0000 7f726c1bd700 1 Processor -- start 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.812+0000 7f726c1bd700 1 -- start start 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f726c1bd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f726419be90 con 0x7f72641043d0 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f7269f59700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72641043d0 0x7f726419b780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43892/0 (socket says 172.21.15.70:43892) 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f7269f59700 1 -- 172.21.15.70:0/2044613783 learned_addr learned my addr 172.21.15.70:0/2044613783 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632383381 0 0) 0x7f726419be90 con 0x7f72641043d0 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72500054f0 con 0x7f72641043d0 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 980697403 0 0) 0x7f72500054f0 con 0x7f72641043d0 2024-02-10T07:33:03.607 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f726419be90 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f7260001400 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2588323448 0 0) 0x7f726419be90 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726419c020 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726419c2d0 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f726419fe20 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f7260004dc0 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.813+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f7260006490 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.814+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 7) v1 ==== 45791+0+0 (unknown 428442895 0 0) 0x7f72600119c0 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.814+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7248005150 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.814+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7f72600474a0 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.819+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f7260013230 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:02.827+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7f7260046010 con 0x7f72641043d0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.032+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 --> v1:172.21.15.70:6800/1990687880 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7248000bc0 con 0x7f7250039fa0 2024-02-10T07:33:03.608 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.037+0000 7f725affd700 1 -- 172.21.15.70:0/2044613783 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f7248000bc0 con 0x7f7250039fa0 2024-02-10T07:33:03.609 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.039+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 >> v1:172.21.15.70:6800/1990687880 conn(0x7f7250039fa0 legacy=0x7f725003c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:03.609 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.039+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 >> v1:172.21.15.70:6789/0 conn(0x7f72641043d0 legacy=0x7f726419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:03.609 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.039+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 shutdown_connections 2024-02-10T07:33:03.609 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.039+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 >> 172.21.15.70:0/2044613783 conn(0x7f72640fffd0 msgr2=0x7f7264192530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:03.609 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.039+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 shutdown_connections 2024-02-10T07:33:03.609 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.039+0000 7f726c1bd700 1 -- 172.21.15.70:0/2044613783 wait complete. 2024-02-10T07:33:03.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:03 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:02] ENGINE Serving on https://172.21.15.70:7150 2024-02-10T07:33:03.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:03 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:02] ENGINE Serving on http://172.21.15.70:8765 2024-02-10T07:33:03.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:03 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:02] ENGINE Bus STARTED 2024-02-10T07:33:03.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:03 smithi070 ceph-mon[29548]: mgrmap e8: y(active, since 2s) 2024-02-10T07:33:03.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:03 smithi070 ceph-mon[29548]: from='client.14132 v1:172.21.15.70:0/2044613783' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:03.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:03 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:03.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:03 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.974+0000 7fc58c306700 1 Processor -- start 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.974+0000 7fc58c306700 1 -- start start 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc58c306700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc584108e40 con 0x7fc584104a10 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc58a0a2700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc584104a10 0x7fc584104e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43908/0 (socket says 172.21.15.70:43908) 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc58a0a2700 1 -- 172.21.15.70:0/1025410313 learned_addr learned my addr 172.21.15.70:0/1025410313 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc5890a0700 1 -- 172.21.15.70:0/1025410313 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357485564 0 0) 0x7fc584108e40 con 0x7fc584104a10 2024-02-10T07:33:04.679 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc5890a0700 1 -- 172.21.15.70:0/1025410313 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5700057c0 con 0x7fc584104a10 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc5890a0700 1 -- 172.21.15.70:0/1025410313 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913910206 0 0) 0x7fc5700057c0 con 0x7fc584104a10 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc5890a0700 1 -- 172.21.15.70:0/1025410313 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc584109fe0 con 0x7fc584104a10 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc5890a0700 1 -- 172.21.15.70:0/1025410313 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc574001580 con 0x7fc584104a10 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc5890a0700 1 -- 172.21.15.70:0/1025410313 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fc574001ce0 con 0x7fc584104a10 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.975+0000 7fc5890a0700 1 -- 172.21.15.70:0/1025410313 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc574005960 con 0x7fc584104a10 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.976+0000 7fc58c306700 1 -- 172.21.15.70:0/1025410313 >> v1:172.21.15.70:6789/0 conn(0x7fc584104a10 legacy=0x7fc584104e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.976+0000 7fc58c306700 1 -- 172.21.15.70:0/1025410313 shutdown_connections 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.976+0000 7fc58c306700 1 -- 172.21.15.70:0/1025410313 >> 172.21.15.70:0/1025410313 conn(0x7fc5840fffd0 msgr2=0x7fc5841023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.976+0000 7fc58c306700 1 -- 172.21.15.70:0/1025410313 shutdown_connections 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.976+0000 7fc58c306700 1 -- 172.21.15.70:0/1025410313 wait complete. 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc58c306700 1 Processor -- start 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc58c306700 1 -- start start 2024-02-10T07:33:04.680 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc58c306700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc584197ae0 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc58a0a2700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc584104a10 0x7fc5841973d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43912/0 (socket says 172.21.15.70:43912) 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc58a0a2700 1 -- 172.21.15.70:0/156027660 learned_addr learned my addr 172.21.15.70:0/156027660 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579893857 0 0) 0x7fc584197ae0 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc570007700 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3962725352 0 0) 0x7fc570007700 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc584197ae0 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc574005940 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1741762841 0 0) 0x7fc584197ae0 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc584197c70 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc584197f20 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.977+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58419ba20 con 0x7fc584104a10 2024-02-10T07:33:04.681 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.978+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fc5740049d0 con 0x7fc584104a10 2024-02-10T07:33:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.978+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc574006010 con 0x7fc584104a10 2024-02-10T07:33:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.978+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7fc5740114d0 con 0x7fc584104a10 2024-02-10T07:33:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.979+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc568005150 con 0x7fc584104a10 2024-02-10T07:33:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.979+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7fc574047180 con 0x7fc584104a10 2024-02-10T07:33:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:03.983+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc5740167c0 con 0x7fc584104a10 2024-02-10T07:33:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.177+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 --> v1:172.21.15.70:6800/1990687880 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc568000bc0 con 0x7fc57003c3d0 2024-02-10T07:33:04.682 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.178+0000 7fc57b7fe700 1 -- 172.21.15.70:0/156027660 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (unknown 0 0 2070689548) 0x7fc568000bc0 con 0x7fc57003c3d0 2024-02-10T07:33:04.683 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.180+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 >> v1:172.21.15.70:6800/1990687880 conn(0x7fc57003c3d0 legacy=0x7fc57003e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:04.683 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.180+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 >> v1:172.21.15.70:6789/0 conn(0x7fc584104a10 legacy=0x7fc5841973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:04.683 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.180+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 shutdown_connections 2024-02-10T07:33:04.683 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.180+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 >> 172.21.15.70:0/156027660 conn(0x7fc5840fffd0 msgr2=0x7fc5841023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:04.683 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.180+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 shutdown_connections 2024-02-10T07:33:04.683 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:04.180+0000 7fc58c306700 1 -- 172.21.15.70:0/156027660 wait complete. 2024-02-10T07:33:04.683 INFO:teuthology.orchestra.run.smithi070.stdout:Generating ssh key... 2024-02-10T07:33:05.437 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-mon[29548]: from='client.14134 v1:172.21.15.70:0/156027660' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:05.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: Generating public/private rsa key pair. 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: Your identification has been saved in /tmp/tmpq3o46y5w/key. 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: Your public key has been saved in /tmp/tmpq3o46y5w/key.pub. 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: The key fingerprint is: 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: SHA256:eGIU5OBL8Qwy15mikk+IYMej4GPElXf5MhNqLVUrMyE ceph-70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: The key's randomart image is: 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: +---[RSA 3072]----+ 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: |. =.*Eo+o. | 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: |o+ X.O=*. . | 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: |Boo.=.X+o. | 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: |==o. * *+. | 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: |.+. o = S | 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: | . . o | 2024-02-10T07:33:05.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: | | 2024-02-10T07:33:05.743 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: | | 2024-02-10T07:33:05.743 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: | | 2024-02-10T07:33:05.743 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: +----[SHA256]-----+ 2024-02-10T07:33:06.002 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.089+0000 7fafba01f700 1 Processor -- start 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.089+0000 7fafba01f700 1 -- start start 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.089+0000 7fafba01f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fafb40796c0 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.089+0000 7fafb37fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fafb4106620 0x7fafb4108a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43918/0 (socket says 172.21.15.70:43918) 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.089+0000 7fafb37fe700 1 -- 172.21.15.70:0/2090923764 learned_addr learned my addr 172.21.15.70:0/2090923764 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafb2ffd700 1 -- 172.21.15.70:0/2090923764 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467425543 0 0) 0x7fafb40796c0 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafb2ffd700 1 -- 172.21.15.70:0/2090923764 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa00035b0 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafb2ffd700 1 -- 172.21.15.70:0/2090923764 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3755048961 0 0) 0x7fafa00035b0 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafb2ffd700 1 -- 172.21.15.70:0/2090923764 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb410b140 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafb2ffd700 1 -- 172.21.15.70:0/2090923764 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fafa4001580 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafb2ffd700 1 -- 172.21.15.70:0/2090923764 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fafa4001ce0 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafb2ffd700 1 -- 172.21.15.70:0/2090923764 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fafa4005960 con 0x7fafb4106620 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.090+0000 7fafba01f700 1 -- 172.21.15.70:0/2090923764 >> v1:172.21.15.70:6789/0 conn(0x7fafb4106620 legacy=0x7fafb4108a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.091+0000 7fafba01f700 1 -- 172.21.15.70:0/2090923764 shutdown_connections 2024-02-10T07:33:06.003 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.091+0000 7fafba01f700 1 -- 172.21.15.70:0/2090923764 >> 172.21.15.70:0/2090923764 conn(0x7fafb40fffd0 msgr2=0x7fafb41023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.091+0000 7fafba01f700 1 -- 172.21.15.70:0/2090923764 shutdown_connections 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.091+0000 7fafba01f700 1 -- 172.21.15.70:0/2090923764 wait complete. 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.091+0000 7fafba01f700 1 Processor -- start 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.091+0000 7fafba01f700 1 -- start start 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafba01f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fafb410afb0 con 0x7fafb4106620 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb37fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fafb4106620 0x7fafb4199570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43932/0 (socket says 172.21.15.70:43932) 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb37fe700 1 -- 172.21.15.70:0/1774719996 learned_addr learned my addr 172.21.15.70:0/1774719996 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451241957 0 0) 0x7fafb410afb0 con 0x7fafb4106620 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa00054f0 con 0x7fafb4106620 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 795731416 0 0) 0x7fafa00054f0 con 0x7fafb4106620 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafb410afb0 con 0x7fafb4106620 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fafa4001080 con 0x7fafb4106620 2024-02-10T07:33:06.004 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2701959430 0 0) 0x7fafb410afb0 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4199c80 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb4199f30 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb419a1e0 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fafa4004a40 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.092+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fafa4006120 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.093+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7fafa40115e0 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.093+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf94005120 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.094+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7fafa40472a0 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.098+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fafa4012e40 con 0x7fafb4106620 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.294+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 --> v1:172.21.15.70:6800/1990687880 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7faf94000b90 con 0x7fafa003a120 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.443+0000 7fafb17fa700 1 -- 172.21.15.70:0/1774719996 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7faf94000b90 con 0x7fafa003a120 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.445+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 >> v1:172.21.15.70:6800/1990687880 conn(0x7fafa003a120 legacy=0x7fafa003c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.445+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 >> v1:172.21.15.70:6789/0 conn(0x7fafb4106620 legacy=0x7fafb4199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:06.005 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.446+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 shutdown_connections 2024-02-10T07:33:06.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.446+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 >> 172.21.15.70:0/1774719996 conn(0x7fafb40fffd0 msgr2=0x7fafb4190350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:06.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.446+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 shutdown_connections 2024-02-10T07:33:06.006 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:05.446+0000 7fafba01f700 1 -- 172.21.15.70:0/1774719996 wait complete. 2024-02-10T07:33:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:06 smithi070 ceph-mon[29548]: from='client.14136 v1:172.21.15.70:0/1774719996' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:06 smithi070 ceph-mon[29548]: Generating ssh key... 2024-02-10T07:33:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:06 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:06 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:07.133 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCFv4C86/2hWBdnb2iRbuWscd4ji35DL3uSGDYPlogdigDJid7acqEI6Z/GFfxvYF270nd0Ol6yLRfktqzz/1squKPCzIg8R0Ms1Rqodpu01sjCzjNWk0YVJRYxpE1K1PN02QPA52xJdL/c6KiAqLkHOL/bZ3PDlngiShuiiy7/yeP0yOyWGKrSeXup2m5EbsQwGT/os4bbDS4KWX81mhL6QpkSsHV+ebu65pqoTTng8KyCYW6BmdxvSErU1GsXc3mGpmbN1dICgt/IY+eoPf0jlFBwJrXRMKayoz8wzPyh9h1Qlrl8j/NMXufsDunhPnXv0ZSz6vAy9VpwXxq/F5w4Blht5PsIMC8rUB29RaLPNC0OcDZc4JqWql1/9md40SOzAFzkDVURK+xJeseohDXGuYdC9lORIkc4mN7ZqXtLL3mBKDqEajziHgvWk9tIEraXPPcFaPqzvq1pZ99z6N7vfEBQ3u1iPHmdCetCD9iBFI0ZaD57cdcPqjDuP3slO0= ceph-70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.413+0000 7f39a4747700 1 Processor -- start 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.413+0000 7f39a4747700 1 -- start start 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a4747700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f399c108ee0 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a24e3700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f399c106100 0x7f399c106520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43934/0 (socket says 172.21.15.70:43934) 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a24e3700 1 -- 172.21.15.70:0/1291671665 learned_addr learned my addr 172.21.15.70:0/1291671665 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a14e1700 1 -- 172.21.15.70:0/1291671665 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287863769 0 0) 0x7f399c108ee0 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a14e1700 1 -- 172.21.15.70:0/1291671665 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39880035b0 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a14e1700 1 -- 172.21.15.70:0/1291671665 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1988991609 0 0) 0x7f39880035b0 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a14e1700 1 -- 172.21.15.70:0/1291671665 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c10a080 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a14e1700 1 -- 172.21.15.70:0/1291671665 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3998001580 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a14e1700 1 -- 172.21.15.70:0/1291671665 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f3998001ce0 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.414+0000 7f39a14e1700 1 -- 172.21.15.70:0/1291671665 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3998005960 con 0x7f399c106100 2024-02-10T07:33:07.134 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.415+0000 7f39a4747700 1 -- 172.21.15.70:0/1291671665 >> v1:172.21.15.70:6789/0 conn(0x7f399c106100 legacy=0x7f399c106520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.415+0000 7f39a4747700 1 -- 172.21.15.70:0/1291671665 shutdown_connections 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.415+0000 7f39a4747700 1 -- 172.21.15.70:0/1291671665 >> 172.21.15.70:0/1291671665 conn(0x7f399c0fffe0 msgr2=0x7f399c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.415+0000 7f39a4747700 1 -- 172.21.15.70:0/1291671665 shutdown_connections 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.415+0000 7f39a4747700 1 -- 172.21.15.70:0/1291671665 wait complete. 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.415+0000 7f39a4747700 1 Processor -- start 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39a4747700 1 -- start start 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39a4747700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f399c199da0 con 0x7f399c106100 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39a24e3700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f399c106100 0x7f399c199690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43942/0 (socket says 172.21.15.70:43942) 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39a24e3700 1 -- 172.21.15.70:0/576470922 learned_addr learned my addr 172.21.15.70:0/576470922 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599590980 0 0) 0x7f399c199da0 con 0x7f399c106100 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39880054f0 con 0x7f399c106100 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2523481328 0 0) 0x7f39880054f0 con 0x7f399c106100 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f399c199da0 con 0x7f399c106100 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3998004870 con 0x7f399c106100 2024-02-10T07:33:07.135 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4249524645 0 0) 0x7f399c199da0 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c199f30 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.416+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c19a1e0 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.417+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c19dd30 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.417+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f3998004a40 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.417+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3998006030 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.417+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7f39980114f0 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.417+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3980005150 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.418+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7f3998047240 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.422+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f3998012d50 con 0x7f399c106100 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.617+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 --> v1:172.21.15.70:6800/1990687880 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3980000bc0 con 0x7f398803a1c0 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.618+0000 7f39937fe700 1 -- 172.21.15.70:0/576470922 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (unknown 0 0 3394994608) 0x7f3980000bc0 con 0x7f398803a1c0 2024-02-10T07:33:07.136 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.620+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 >> v1:172.21.15.70:6800/1990687880 conn(0x7f398803a1c0 legacy=0x7f398803c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.620+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 >> v1:172.21.15.70:6789/0 conn(0x7f399c106100 legacy=0x7f399c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.620+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 shutdown_connections 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.620+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 >> 172.21.15.70:0/576470922 conn(0x7f399c0fffe0 msgr2=0x7f399c078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.620+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 shutdown_connections 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:06.620+0000 7f39a4747700 1 -- 172.21.15.70:0/576470922 wait complete. 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:Adding key to root@localhost authorized_keys... 2024-02-10T07:33:07.137 INFO:teuthology.orchestra.run.smithi070.stdout:Adding host smithi070... 2024-02-10T07:33:07.740 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:07 smithi070 ceph-mon[29548]: from='client.14138 v1:172.21.15.70:0/576470922' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:08.695 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:08 smithi070 ceph-mon[29548]: from='client.14140 v1:172.21.15.70:0/3250819496' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:09 smithi070 ceph-mon[29548]: Deploying cephadm binary to smithi070 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Added host 'smithi070' with addr '172.21.15.70' 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.533+0000 7f293adef700 1 Processor -- start 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f293adef700 1 -- start start 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f293adef700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2934108e30 con 0x7f2934104a00 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f2938b8b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2934104a00 0x7f2934104e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41288/0 (socket says 172.21.15.70:41288) 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f2938b8b700 1 -- 172.21.15.70:0/2659464068 learned_addr learned my addr 172.21.15.70:0/2659464068 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f29337fe700 1 -- 172.21.15.70:0/2659464068 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953668465 0 0) 0x7f2934108e30 con 0x7f2934104a00 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f29337fe700 1 -- 172.21.15.70:0/2659464068 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29240035b0 con 0x7f2934104a00 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f29337fe700 1 -- 172.21.15.70:0/2659464068 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4186009720 0 0) 0x7f29240035b0 con 0x7f2934104a00 2024-02-10T07:33:12.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f29337fe700 1 -- 172.21.15.70:0/2659464068 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934109fd0 con 0x7f2934104a00 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f29337fe700 1 -- 172.21.15.70:0/2659464068 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f2928001580 con 0x7f2934104a00 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f29337fe700 1 -- 172.21.15.70:0/2659464068 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f2928001ce0 con 0x7f2934104a00 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.534+0000 7f29337fe700 1 -- 172.21.15.70:0/2659464068 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f2928005960 con 0x7f2934104a00 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.535+0000 7f293adef700 1 -- 172.21.15.70:0/2659464068 >> v1:172.21.15.70:6789/0 conn(0x7f2934104a00 legacy=0x7f2934104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.535+0000 7f293adef700 1 -- 172.21.15.70:0/2659464068 shutdown_connections 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.535+0000 7f293adef700 1 -- 172.21.15.70:0/2659464068 >> 172.21.15.70:0/2659464068 conn(0x7f29340fffe0 msgr2=0x7f2934102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.535+0000 7f293adef700 1 -- 172.21.15.70:0/2659464068 shutdown_connections 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.535+0000 7f293adef700 1 -- 172.21.15.70:0/2659464068 wait complete. 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f293adef700 1 Processor -- start 2024-02-10T07:33:12.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f293adef700 1 -- start start 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f293adef700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f293419f590 con 0x7f2934104a00 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2938b8b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2934104a00 0x7f2934101e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41304/0 (socket says 172.21.15.70:41304) 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2938b8b700 1 -- 172.21.15.70:0/3250819496 learned_addr learned my addr 172.21.15.70:0/3250819496 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125249551 0 0) 0x7f293419f590 con 0x7f2934104a00 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29240054f0 con 0x7f2934104a00 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3555229815 0 0) 0x7f29240054f0 con 0x7f2934104a00 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f293419f590 con 0x7f2934104a00 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f2928005940 con 0x7f2934104a00 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.536+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 187355613 0 0) 0x7f293419f590 con 0x7f2934104a00 2024-02-10T07:33:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.537+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29341a0730 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.537+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293419f780 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.537+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f29280052d0 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.537+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f29280061f0 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.537+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f293419fc80 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.537+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7f2928006350 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.538+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7f29280472d0 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.538+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2934063350 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.543+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f2928007bc0 con 0x7f2934104a00 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:07.739+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 --> v1:172.21.15.70:6800/1990687880 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}) v1 -- 0x7f29341a01f0 con 0x7f292403a170 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:11.904+0000 7f2931ffb700 1 -- 172.21.15.70:0/3250819496 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (unknown 0 0 1740527350) 0x7f29341a01f0 con 0x7f292403a170 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:11.907+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 >> v1:172.21.15.70:6800/1990687880 conn(0x7f292403a170 legacy=0x7f292403c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:11.907+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 >> v1:172.21.15.70:6789/0 conn(0x7f2934104a00 legacy=0x7f2934101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:11.907+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 shutdown_connections 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:11.907+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 >> 172.21.15.70:0/3250819496 conn(0x7f29340fffe0 msgr2=0x7f29340788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:12.477 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:11.907+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 shutdown_connections 2024-02-10T07:33:12.478 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:11.907+0000 7f293adef700 1 -- 172.21.15.70:0/3250819496 wait complete. 2024-02-10T07:33:12.478 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying unmanaged mon service... 2024-02-10T07:33:13.096 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:12 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:13.096 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:12 smithi070 ceph-mon[29548]: Added host smithi070 2024-02-10T07:33:13.096 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:12 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-10T07:33:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.879+0000 7f5bad368700 1 Processor -- start 2024-02-10T07:33:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.879+0000 7f5bad368700 1 -- start start 2024-02-10T07:33:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.879+0000 7f5bad368700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ba8079700 con 0x7f5ba81065d0 2024-02-10T07:33:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.879+0000 7f5ba6ffd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5ba81065d0 0x7f5ba81089c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41312/0 (socket says 172.21.15.70:41312) 2024-02-10T07:33:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.879+0000 7f5ba6ffd700 1 -- 172.21.15.70:0/461936631 learned_addr learned my addr 172.21.15.70:0/461936631 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:14.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5ba5ffb700 1 -- 172.21.15.70:0/461936631 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911620037 0 0) 0x7f5ba8079700 con 0x7f5ba81065d0 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5ba5ffb700 1 -- 172.21.15.70:0/461936631 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b900035b0 con 0x7f5ba81065d0 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5ba5ffb700 1 -- 172.21.15.70:0/461936631 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801799104 0 0) 0x7f5b900035b0 con 0x7f5ba81065d0 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5ba5ffb700 1 -- 172.21.15.70:0/461936631 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba810b0f0 con 0x7f5ba81065d0 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5ba5ffb700 1 -- 172.21.15.70:0/461936631 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5b98001580 con 0x7f5ba81065d0 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5ba5ffb700 1 -- 172.21.15.70:0/461936631 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f5b98001ce0 con 0x7f5ba81065d0 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5ba5ffb700 1 -- 172.21.15.70:0/461936631 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5b98005960 con 0x7f5ba81065d0 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5bad368700 1 -- 172.21.15.70:0/461936631 >> v1:172.21.15.70:6789/0 conn(0x7f5ba81065d0 legacy=0x7f5ba81089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5bad368700 1 -- 172.21.15.70:0/461936631 shutdown_connections 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.880+0000 7f5bad368700 1 -- 172.21.15.70:0/461936631 >> 172.21.15.70:0/461936631 conn(0x7f5ba80fffe0 msgr2=0x7f5ba8102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.881+0000 7f5bad368700 1 -- 172.21.15.70:0/461936631 shutdown_connections 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.881+0000 7f5bad368700 1 -- 172.21.15.70:0/461936631 wait complete. 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.881+0000 7f5bad368700 1 Processor -- start 2024-02-10T07:33:14.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.881+0000 7f5bad368700 1 -- start start 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.881+0000 7f5bad368700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5ba8199c80 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.881+0000 7f5ba6ffd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5ba81065d0 0x7f5ba8199570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41322/0 (socket says 172.21.15.70:41322) 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.881+0000 7f5ba6ffd700 1 -- 172.21.15.70:0/2983354142 learned_addr learned my addr 172.21.15.70:0/2983354142 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942789664 0 0) 0x7f5ba8199c80 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b900054f0 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1931999821 0 0) 0x7f5b900054f0 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ba8199c80 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5b98001080 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 471266233 0 0) 0x7f5ba8199c80 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8199e10 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba819a0c0 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba819dc10 con 0x7f5ba81065d0 2024-02-10T07:33:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7f5b98004a40 con 0x7f5ba81065d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.882+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f5b98006110 con 0x7f5ba81065d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.883+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7f5b980116b0 con 0x7f5ba81065d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.883+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b94005120 con 0x7f5ba81065d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.883+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7f5b98047390 con 0x7f5ba81065d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:12.888+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f5b98012f20 con 0x7f5ba81065d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.095+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 --> v1:172.21.15.70:6800/1990687880 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5b94000b90 con 0x7f5b9003a0d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.100+0000 7f5b8ffff700 1 -- 172.21.15.70:0/2983354142 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f5b94000b90 con 0x7f5b9003a0d0 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.102+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 >> v1:172.21.15.70:6800/1990687880 conn(0x7f5b9003a0d0 legacy=0x7f5b9003c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.102+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 >> v1:172.21.15.70:6789/0 conn(0x7f5ba81065d0 legacy=0x7f5ba8199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.102+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 shutdown_connections 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.102+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 >> 172.21.15.70:0/2983354142 conn(0x7f5ba80fffe0 msgr2=0x7f5ba81903b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.102+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 shutdown_connections 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:13.102+0000 7f5bad368700 1 -- 172.21.15.70:0/2983354142 wait complete. 2024-02-10T07:33:14.297 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying unmanaged mgr service... 2024-02-10T07:33:14.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:14 smithi070 ceph-mon[29548]: from='client.14142 v1:172.21.15.70:0/2983354142' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:14.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:14 smithi070 ceph-mon[29548]: Saving service mon spec with placement count:5 2024-02-10T07:33:14.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:14 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.688+0000 7fede9f32700 1 Processor -- start 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede9f32700 1 -- start start 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede9f32700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fede4079720 con 0x7fede41065b0 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede37fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fede41065b0 0x7fede41089a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41336/0 (socket says 172.21.15.70:41336) 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede37fe700 1 -- 172.21.15.70:0/3409858665 learned_addr learned my addr 172.21.15.70:0/3409858665 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede27fc700 1 -- 172.21.15.70:0/3409858665 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539955972 0 0) 0x7fede4079720 con 0x7fede41065b0 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede27fc700 1 -- 172.21.15.70:0/3409858665 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedcc0035b0 con 0x7fede41065b0 2024-02-10T07:33:15.719 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede27fc700 1 -- 172.21.15.70:0/3409858665 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790623853 0 0) 0x7fedcc0035b0 con 0x7fede41065b0 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede27fc700 1 -- 172.21.15.70:0/3409858665 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede410b0d0 con 0x7fede41065b0 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede27fc700 1 -- 172.21.15.70:0/3409858665 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fedd8001580 con 0x7fede41065b0 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.689+0000 7fede27fc700 1 -- 172.21.15.70:0/3409858665 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fedd8001ce0 con 0x7fede41065b0 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.690+0000 7fede27fc700 1 -- 172.21.15.70:0/3409858665 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fedd8005960 con 0x7fede41065b0 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.690+0000 7fede9f32700 1 -- 172.21.15.70:0/3409858665 >> v1:172.21.15.70:6789/0 conn(0x7fede41065b0 legacy=0x7fede41089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.690+0000 7fede9f32700 1 -- 172.21.15.70:0/3409858665 shutdown_connections 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.690+0000 7fede9f32700 1 -- 172.21.15.70:0/3409858665 >> 172.21.15.70:0/3409858665 conn(0x7fede40fffe0 msgr2=0x7fede4102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.690+0000 7fede9f32700 1 -- 172.21.15.70:0/3409858665 shutdown_connections 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.690+0000 7fede9f32700 1 -- 172.21.15.70:0/3409858665 wait complete. 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede9f32700 1 Processor -- start 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede9f32700 1 -- start start 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede9f32700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fede4199c80 con 0x7fede41065b0 2024-02-10T07:33:15.720 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede37fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fede41065b0 0x7fede4199570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41346/0 (socket says 172.21.15.70:41346) 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede37fe700 1 -- 172.21.15.70:0/1874261543 learned_addr learned my addr 172.21.15.70:0/1874261543 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556714159 0 0) 0x7fede4199c80 con 0x7fede41065b0 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedcc0054f0 con 0x7fede41065b0 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.691+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1160160172 0 0) 0x7fedcc0054f0 con 0x7fede41065b0 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fede4199c80 con 0x7fede41065b0 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fedd8001080 con 0x7fede41065b0 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4003467264 0 0) 0x7fede4199c80 con 0x7fede41065b0 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede4199e10 con 0x7fede41065b0 2024-02-10T07:33:15.721 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede419a0c0 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fede419dc10 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 742557412 0 0) 0x7fedd8004a40 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.692+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fedd8006120 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.693+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7fedd80116a0 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.693+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd0005120 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.693+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7fedd8047360 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.698+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fedd8012f10 con 0x7fede41065b0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.905+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 --> v1:172.21.15.70:6800/1990687880 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fedd0000b90 con 0x7fedcc03a0d0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.909+0000 7fede0ff9700 1 -- 172.21.15.70:0/1874261543 <== mgr.14120 v1:172.21.15.70:6800/1990687880 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7fedd0000b90 con 0x7fedcc03a0d0 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.911+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 >> v1:172.21.15.70:6800/1990687880 conn(0x7fedcc03a0d0 legacy=0x7fedcc03c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.911+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 >> v1:172.21.15.70:6789/0 conn(0x7fede41065b0 legacy=0x7fede4199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.911+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 shutdown_connections 2024-02-10T07:33:15.722 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.911+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 >> 172.21.15.70:0/1874261543 conn(0x7fede40fffe0 msgr2=0x7fede4190350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:15.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.911+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 shutdown_connections 2024-02-10T07:33:15.723 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:14.911+0000 7fede9f32700 1 -- 172.21.15.70:0/1874261543 wait complete. 2024-02-10T07:33:15.977 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:15 smithi070 ceph-mon[29548]: from='client.14144 v1:172.21.15.70:0/1874261543' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:15.977 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:15 smithi070 ceph-mon[29548]: Saving service mgr spec with placement count:2 2024-02-10T07:33:15.977 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:15 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:15.977 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:15 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:15.978 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:15 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:17.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.177+0000 7fbbbb18f700 1 Processor -- start 2024-02-10T07:33:17.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.177+0000 7fbbbb18f700 1 -- start start 2024-02-10T07:33:17.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.177+0000 7fbbbb18f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbbb4078df0 con 0x7fbbb4106600 2024-02-10T07:33:17.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.177+0000 7fbbb8f2b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbbb4106600 0x7fbbb41089f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41352/0 (socket says 172.21.15.70:41352) 2024-02-10T07:33:17.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.177+0000 7fbbb8f2b700 1 -- 172.21.15.70:0/2230830973 learned_addr learned my addr 172.21.15.70:0/2230830973 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:17.243 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.178+0000 7fbbb37fe700 1 -- 172.21.15.70:0/2230830973 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334055438 0 0) 0x7fbbb4078df0 con 0x7fbbb4106600 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.178+0000 7fbbb37fe700 1 -- 172.21.15.70:0/2230830973 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba40035b0 con 0x7fbbb4106600 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.178+0000 7fbbb37fe700 1 -- 172.21.15.70:0/2230830973 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474322513 0 0) 0x7fbba40035b0 con 0x7fbbb4106600 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.178+0000 7fbbb37fe700 1 -- 172.21.15.70:0/2230830973 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb410b120 con 0x7fbbb4106600 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.178+0000 7fbbb37fe700 1 -- 172.21.15.70:0/2230830973 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbba8001580 con 0x7fbbb4106600 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.178+0000 7fbbb37fe700 1 -- 172.21.15.70:0/2230830973 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fbba8001d00 con 0x7fbbb4106600 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.178+0000 7fbbb37fe700 1 -- 172.21.15.70:0/2230830973 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbba8005960 con 0x7fbbb4106600 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.179+0000 7fbbbb18f700 1 -- 172.21.15.70:0/2230830973 >> v1:172.21.15.70:6789/0 conn(0x7fbbb4106600 legacy=0x7fbbb41089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.179+0000 7fbbbb18f700 1 -- 172.21.15.70:0/2230830973 shutdown_connections 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.179+0000 7fbbbb18f700 1 -- 172.21.15.70:0/2230830973 >> 172.21.15.70:0/2230830973 conn(0x7fbbb40fff70 msgr2=0x7fbbb4102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.179+0000 7fbbbb18f700 1 -- 172.21.15.70:0/2230830973 shutdown_connections 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.179+0000 7fbbbb18f700 1 -- 172.21.15.70:0/2230830973 wait complete. 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.179+0000 7fbbbb18f700 1 Processor -- start 2024-02-10T07:33:17.244 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbbb18f700 1 -- start start 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbbb18f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbbb419f620 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb8f2b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbbb4106600 0x7fbbb4105e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41366/0 (socket says 172.21.15.70:41366) 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb8f2b700 1 -- 172.21.15.70:0/3446955696 learned_addr learned my addr 172.21.15.70:0/3446955696 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917708589 0 0) 0x7fbbb419f620 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba40054f0 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2997420123 0 0) 0x7fbba40054f0 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbbb419f620 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbba8001080 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1351478898 0 0) 0x7fbbb419f620 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb41a07c0 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.180+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb419f810 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.181+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fbba8004a60 con 0x7fbbb4106600 2024-02-10T07:33:17.245 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.181+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbba8006220 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.181+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb419fd10 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.181+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7fbba80063a0 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.182+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7fbba80480f0 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.182+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb4063350 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.186+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fbba8007c10 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.396+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 --> v1:172.21.15.70:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fbbb410acd0 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.398+0000 7fbbb1ffb700 1 -- 172.21.15.70:0/3446955696 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (unknown 2684808024 0 0) 0x7fbba8014f10 con 0x7fbbb4106600 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.401+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 >> v1:172.21.15.70:6800/1990687880 conn(0x7fbba403a170 legacy=0x7fbba403c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.401+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 >> v1:172.21.15.70:6789/0 conn(0x7fbbb4106600 legacy=0x7fbbb4105e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.402+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 shutdown_connections 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.402+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 >> 172.21.15.70:0/3446955696 conn(0x7fbbb40fff70 msgr2=0x7fbbb4190380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.402+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 shutdown_connections 2024-02-10T07:33:17.246 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:16.402+0000 7fbbbb18f700 1 -- 172.21.15.70:0/3446955696 wait complete. 2024-02-10T07:33:17.692 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3446955696' entity='client.admin' 2024-02-10T07:33:18.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:18 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:18.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1673078757' entity='client.admin' 2024-02-10T07:33:18.870 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.009+0000 7f3e37675700 1 Processor -- start 2024-02-10T07:33:18.870 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.009+0000 7f3e37675700 1 -- start start 2024-02-10T07:33:18.870 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.009+0000 7f3e37675700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3e3010a000 con 0x7f3e30103da0 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.009+0000 7f3e35411700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3e30103da0 0x7f3e301041c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44684/0 (socket says 172.21.15.70:44684) 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.009+0000 7f3e35411700 1 -- 172.21.15.70:0/2653780223 learned_addr learned my addr 172.21.15.70:0/2653780223 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.010+0000 7f3e27fff700 1 -- 172.21.15.70:0/2653780223 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736782123 0 0) 0x7f3e3010a000 con 0x7f3e30103da0 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.010+0000 7f3e27fff700 1 -- 172.21.15.70:0/2653780223 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e1c0035b0 con 0x7f3e30103da0 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.010+0000 7f3e27fff700 1 -- 172.21.15.70:0/2653780223 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1328759299 0 0) 0x7f3e1c0035b0 con 0x7f3e30103da0 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.010+0000 7f3e27fff700 1 -- 172.21.15.70:0/2653780223 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3010b1a0 con 0x7f3e30103da0 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.010+0000 7f3e27fff700 1 -- 172.21.15.70:0/2653780223 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3e2c001580 con 0x7f3e30103da0 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.010+0000 7f3e27fff700 1 -- 172.21.15.70:0/2653780223 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f3e2c001d00 con 0x7f3e30103da0 2024-02-10T07:33:18.871 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.010+0000 7f3e27fff700 1 -- 172.21.15.70:0/2653780223 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3e2c005960 con 0x7f3e30103da0 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.011+0000 7f3e37675700 1 -- 172.21.15.70:0/2653780223 >> v1:172.21.15.70:6789/0 conn(0x7f3e30103da0 legacy=0x7f3e301041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.011+0000 7f3e37675700 1 -- 172.21.15.70:0/2653780223 shutdown_connections 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.011+0000 7f3e37675700 1 -- 172.21.15.70:0/2653780223 >> 172.21.15.70:0/2653780223 conn(0x7f3e300ff6f0 msgr2=0x7f3e30101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.011+0000 7f3e37675700 1 -- 172.21.15.70:0/2653780223 shutdown_connections 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.011+0000 7f3e37675700 1 -- 172.21.15.70:0/2653780223 wait complete. 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.011+0000 7f3e37675700 1 Processor -- start 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.011+0000 7f3e37675700 1 -- start start 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e37675700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3e3019f610 con 0x7f3e30103da0 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e35411700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3e30103da0 0x7f3e30106650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44696/0 (socket says 172.21.15.70:44696) 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e35411700 1 -- 172.21.15.70:0/1673078757 learned_addr learned my addr 172.21.15.70:0/1673078757 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609916200 0 0) 0x7f3e3019f610 con 0x7f3e30103da0 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e1c0054f0 con 0x7f3e30103da0 2024-02-10T07:33:18.872 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 807190926 0 0) 0x7f3e1c0054f0 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e3019f610 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3e2c001080 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 638226629 0 0) 0x7f3e3019f610 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e301a07b0 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3019f800 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.012+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3019fd00 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.013+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f3e2c004a60 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.013+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f3e2c006180 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.013+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7f3e2c011740 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.014+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7f3e2c048230 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.014+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e14005150 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.019+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f3e2c012f10 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.223+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 --> v1:172.21.15.70:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f3e14005980 con 0x7f3e30103da0 2024-02-10T07:33:18.873 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.226+0000 7f3e267fc700 1 -- 172.21.15.70:0/1673078757 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (unknown 1496729260 0 0) 0x7f3e2c014ff0 con 0x7f3e30103da0 2024-02-10T07:33:18.874 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.228+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 >> v1:172.21.15.70:6800/1990687880 conn(0x7f3e1c03a170 legacy=0x7f3e1c03c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:18.874 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.228+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 >> v1:172.21.15.70:6789/0 conn(0x7f3e30103da0 legacy=0x7f3e30106650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:18.874 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.228+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 shutdown_connections 2024-02-10T07:33:18.874 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.228+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 >> 172.21.15.70:0/1673078757 conn(0x7f3e300ff6f0 msgr2=0x7f3e30190390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:18.874 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.228+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 shutdown_connections 2024-02-10T07:33:18.874 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:18.228+0000 7f3e37675700 1 -- 172.21.15.70:0/1673078757 wait complete. 2024-02-10T07:33:18.874 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling the dashboard module... 2024-02-10T07:33:20.130 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:19 smithi070 ceph-mon[29548]: from='mgr.14120 v1:172.21.15.70:0/992788223' entity='mgr.y' 2024-02-10T07:33:20.130 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3367079590' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-10T07:33:20.130 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:19 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ignoring --setuser ceph since I am not root 2024-02-10T07:33:20.130 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:19 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ignoring --setgroup ceph since I am not root 2024-02-10T07:33:20.697 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:20 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:20.501+0000 7f5aac236200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:33:20.697 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:20 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:20.594+0000 7f5aac236200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:33:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3367079590' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-10T07:33:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:20 smithi070 ceph-mon[29548]: mgrmap e9: y(active, since 19s) 2024-02-10T07:33:21.064 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.326+0000 7fc5afbf3700 1 Processor -- start 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.327+0000 7fc5afbf3700 1 -- start start 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.327+0000 7fc5afbf3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc5a8109f40 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.327+0000 7fc5ad98f700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5a8105b10 0x7fc5a8105f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44712/0 (socket says 172.21.15.70:44712) 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.327+0000 7fc5ad98f700 1 -- 172.21.15.70:0/4197165305 learned_addr learned my addr 172.21.15.70:0/4197165305 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.327+0000 7fc5ac98d700 1 -- 172.21.15.70:0/4197165305 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139551335 0 0) 0x7fc5a8109f40 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.327+0000 7fc5ac98d700 1 -- 172.21.15.70:0/4197165305 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5940035b0 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5ac98d700 1 -- 172.21.15.70:0/4197165305 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897552198 0 0) 0x7fc5940035b0 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5ac98d700 1 -- 172.21.15.70:0/4197165305 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a810b0e0 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5ac98d700 1 -- 172.21.15.70:0/4197165305 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc5a4001580 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5ac98d700 1 -- 172.21.15.70:0/4197165305 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fc5a4001c30 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5ac98d700 1 -- 172.21.15.70:0/4197165305 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc5a4005770 con 0x7fc5a8105b10 2024-02-10T07:33:21.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5afbf3700 1 -- 172.21.15.70:0/4197165305 >> v1:172.21.15.70:6789/0 conn(0x7fc5a8105b10 legacy=0x7fc5a8105f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5afbf3700 1 -- 172.21.15.70:0/4197165305 shutdown_connections 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.328+0000 7fc5afbf3700 1 -- 172.21.15.70:0/4197165305 >> 172.21.15.70:0/4197165305 conn(0x7fc5a81010d0 msgr2=0x7fc5a81034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.329+0000 7fc5afbf3700 1 -- 172.21.15.70:0/4197165305 shutdown_connections 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.329+0000 7fc5afbf3700 1 -- 172.21.15.70:0/4197165305 wait complete. 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.329+0000 7fc5afbf3700 1 Processor -- start 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.329+0000 7fc5afbf3700 1 -- start start 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.329+0000 7fc5afbf3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc5a819f650 con 0x7fc5a8105b10 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.329+0000 7fc5ad98f700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5a8105b10 0x7fc5a807b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44716/0 (socket says 172.21.15.70:44716) 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.329+0000 7fc5ad98f700 1 -- 172.21.15.70:0/3367079590 learned_addr learned my addr 172.21.15.70:0/3367079590 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961534116 0 0) 0x7fc5a819f650 con 0x7fc5a8105b10 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc594005540 con 0x7fc5a8105b10 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3582992076 0 0) 0x7fc594005540 con 0x7fc5a8105b10 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5a819f650 con 0x7fc5a8105b10 2024-02-10T07:33:21.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc5a4001080 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2100735552 0 0) 0x7fc5a819f650 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a81a07f0 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a819f840 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.330+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a819fd40 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.331+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fc5a40049f0 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.331+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc5a4006260 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.331+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c005150 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.331+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 8) v1 ==== 45897+0+0 (unknown 1625426616 0 0) 0x7fc5a40074f0 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.332+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7fc5a4047310 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.340+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc5a4007d60 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.616+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc58c005400 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.936+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (unknown 1498667528 0 0) 0x7fc5a4016780 con 0x7fc5a8105b10 2024-02-10T07:33:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.937+0000 7fc59effd700 1 -- 172.21.15.70:0/3367079590 <== mon.0 v1:172.21.15.70:6789/0 11 ==== mgrmap(e 9) v1 ==== 45910+0+0 (unknown 3539928795 0 0) 0x7fc5a4045cb0 con 0x7fc5a8105b10 2024-02-10T07:33:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.939+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 >> v1:172.21.15.70:6800/1990687880 conn(0x7fc59403a110 legacy=0x7fc59403c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.939+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 >> v1:172.21.15.70:6789/0 conn(0x7fc5a8105b10 legacy=0x7fc5a807b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.939+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 shutdown_connections 2024-02-10T07:33:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.939+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 >> 172.21.15.70:0/3367079590 conn(0x7fc5a81010d0 msgr2=0x7fc5a8190370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.939+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 shutdown_connections 2024-02-10T07:33:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:19.939+0000 7fc5afbf3700 1 -- 172.21.15.70:0/3367079590 wait complete. 2024-02-10T07:33:21.340 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:21 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:21.244+0000 7f5aac236200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:33:21.340 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:21 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:21.339+0000 7f5aac236200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:33:21.668 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:21 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:21.487+0000 7f5aac236200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:33:21.668 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:21 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:21.584+0000 7f5aac236200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:33:21.936 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:21 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:21.668+0000 7f5aac236200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:33:21.937 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:21 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:21.936+0000 7f5aac236200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:33:22.232 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:22.047+0000 7f5aac236200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:33:22.232 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:22.134+0000 7f5aac236200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:33:22.232 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:22.231+0000 7f5aac236200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.462+0000 7f9ce4982700 1 Processor -- start 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.462+0000 7f9ce4982700 1 -- start start 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.462+0000 7f9ce4982700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9ce0104b40 con 0x7f9ce0106030 2024-02-10T07:33:22.305 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.462+0000 7f9cde59c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9ce0106030 0x7f9ce0106450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44740/0 (socket says 172.21.15.70:44740) 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.462+0000 7f9cde59c700 1 -- 172.21.15.70:0/3676514343 learned_addr learned my addr 172.21.15.70:0/3676514343 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.463+0000 7f9cdd59a700 1 -- 172.21.15.70:0/3676514343 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270495437 0 0) 0x7f9ce0104b40 con 0x7f9ce0106030 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.463+0000 7f9cdd59a700 1 -- 172.21.15.70:0/3676514343 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cc80035b0 con 0x7f9ce0106030 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.463+0000 7f9cdd59a700 1 -- 172.21.15.70:0/3676514343 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639141895 0 0) 0x7f9cc80035b0 con 0x7f9ce0106030 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.463+0000 7f9cdd59a700 1 -- 172.21.15.70:0/3676514343 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce0104cd0 con 0x7f9ce0106030 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.463+0000 7f9cdd59a700 1 -- 172.21.15.70:0/3676514343 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9cd0001580 con 0x7f9ce0106030 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.463+0000 7f9cdd59a700 1 -- 172.21.15.70:0/3676514343 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f9cd0001d00 con 0x7f9ce0106030 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.463+0000 7f9cdd59a700 1 -- 172.21.15.70:0/3676514343 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9cd0005960 con 0x7f9ce0106030 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.464+0000 7f9ce4982700 1 -- 172.21.15.70:0/3676514343 >> v1:172.21.15.70:6789/0 conn(0x7f9ce0106030 legacy=0x7f9ce0106450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.464+0000 7f9ce4982700 1 -- 172.21.15.70:0/3676514343 shutdown_connections 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.464+0000 7f9ce4982700 1 -- 172.21.15.70:0/3676514343 >> 172.21.15.70:0/3676514343 conn(0x7f9ce00ff6f0 msgr2=0x7f9ce0101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.464+0000 7f9ce4982700 1 -- 172.21.15.70:0/3676514343 shutdown_connections 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.464+0000 7f9ce4982700 1 -- 172.21.15.70:0/3676514343 wait complete. 2024-02-10T07:33:22.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.465+0000 7f9ce4982700 1 Processor -- start 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.465+0000 7f9ce4982700 1 -- start start 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.465+0000 7f9ce4982700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9ce019beb0 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.465+0000 7f9cde59c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9ce0106030 0x7f9ce019b7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44742/0 (socket says 172.21.15.70:44742) 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.465+0000 7f9cde59c700 1 -- 172.21.15.70:0/876041588 learned_addr learned my addr 172.21.15.70:0/876041588 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.465+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178840673 0 0) 0x7f9ce019beb0 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.465+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cc80054f0 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3564339062 0 0) 0x7f9cc80054f0 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ce019beb0 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9cd0001080 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 141240310 0 0) 0x7f9ce019beb0 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce019c040 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce019c2f0 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce019fe40 con 0x7f9ce0106030 2024-02-10T07:33:22.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f9cd0001870 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.466+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f9cd00061c0 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.467+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 9) v1 ==== 45910+0+0 (unknown 3539928795 0 0) 0x7f9cd0011760 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.467+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc0005150 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.467+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7f9cd00482e0 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.467+0000 7f9cddd9b700 1 -- 172.21.15.70:0/876041588 >> v1:172.21.15.70:6800/1990687880 conn(0x7f9cc803a120 legacy=0x7f9cc803c5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1990687880 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.472+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f9cd0013020 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.668+0000 7f9cddd9b700 1 -- 172.21.15.70:0/876041588 >> v1:172.21.15.70:6800/1990687880 conn(0x7f9cc803a120 legacy=0x7f9cc803c5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1990687880 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.728+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9cc0005c10 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.728+0000 7f9ccf7fe700 1 -- 172.21.15.70:0/876041588 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f9cd0015100 con 0x7f9ce0106030 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.730+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 >> v1:172.21.15.70:6800/1990687880 conn(0x7f9cc803a120 legacy=0x7f9cc803c5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.730+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 >> v1:172.21.15.70:6789/0 conn(0x7f9ce0106030 legacy=0x7f9ce019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.730+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 shutdown_connections 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.730+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 >> 172.21.15.70:0/876041588 conn(0x7f9ce00ff6f0 msgr2=0x7f9ce01925d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:22.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.730+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 shutdown_connections 2024-02-10T07:33:22.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:21.730+0000 7f9ce4982700 1 -- 172.21.15.70:0/876041588 wait complete. 2024-02-10T07:33:22.309 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-02-10T07:33:22.309 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 9... 2024-02-10T07:33:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:22.412+0000 7f5aac236200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:33:22.790 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/876041588' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T07:33:23.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:23 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:23.385+0000 7f5aac236200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:33:23.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:23 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:23.517+0000 7f5aac236200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:33:24.134 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:23 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:23.850+0000 7f5aac236200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:33:24.135 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:23 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:23.940+0000 7f5aac236200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:33:24.135 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:24.134+0000 7f5aac236200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:33:24.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:24.320+0000 7f5aac236200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:33:24.862 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:24.604+0000 7f5aac236200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:33:25.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:24.861+0000 7f5aac236200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:33:25.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:25 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:25.096+0000 7f5aac236200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:33:26.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:25 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:25.895+0000 7f5aac236200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:33:26.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:26 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:26.464+0000 7f5aac236200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:33:26.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:26 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:26.562+0000 7f5aac236200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:33:27.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:33:26 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:33:26.777+0000 7f5aac236200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:33:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: Active manager daemon y restarted 2024-02-10T07:33:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: Activating manager daemon y 2024-02-10T07:33:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: osdmap e3: 0 total, 0 up, 0 in 2024-02-10T07:33:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: mgrmap e10: y(active, starting, since 0.00320377s) 2024-02-10T07:33:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: Manager daemon y is now available 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:33:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:33:28.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:33:28.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-10T07:33:28.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T07:33:28.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:33:28.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.766+0000 7f18ef296700 1 Processor -- start 2024-02-10T07:33:28.714 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.766+0000 7f18ef296700 1 -- start start 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.766+0000 7f18ef296700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f18e8079760 con 0x7f18e8104370 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.773+0000 7f18ed032700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18e8104370 0x7f18e81067a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44754/0 (socket says 172.21.15.70:44754) 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.773+0000 7f18ed032700 1 -- 172.21.15.70:0/2547283299 learned_addr learned my addr 172.21.15.70:0/2547283299 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.802+0000 7f18dffff700 1 -- 172.21.15.70:0/2547283299 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464979453 0 0) 0x7f18e8079760 con 0x7f18e8104370 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.802+0000 7f18dffff700 1 -- 172.21.15.70:0/2547283299 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d40035b0 con 0x7f18e8104370 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.815+0000 7f18dffff700 1 -- 172.21.15.70:0/2547283299 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1533754491 0 0) 0x7f18d40035b0 con 0x7f18e8104370 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.815+0000 7f18dffff700 1 -- 172.21.15.70:0/2547283299 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8108ed0 con 0x7f18e8104370 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.815+0000 7f18dffff700 1 -- 172.21.15.70:0/2547283299 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f18e0001900 con 0x7f18e8104370 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.815+0000 7f18dffff700 1 -- 172.21.15.70:0/2547283299 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f18e0001fb0 con 0x7f18e8104370 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.816+0000 7f18ef296700 1 -- 172.21.15.70:0/2547283299 >> v1:172.21.15.70:6789/0 conn(0x7f18e8104370 legacy=0x7f18e81067a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.816+0000 7f18ef296700 1 -- 172.21.15.70:0/2547283299 shutdown_connections 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.816+0000 7f18ef296700 1 -- 172.21.15.70:0/2547283299 >> 172.21.15.70:0/2547283299 conn(0x7f18e80fffd0 msgr2=0x7f18e81023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.817+0000 7f18ef296700 1 -- 172.21.15.70:0/2547283299 shutdown_connections 2024-02-10T07:33:28.715 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.817+0000 7f18ef296700 1 -- 172.21.15.70:0/2547283299 wait complete. 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.817+0000 7f18ef296700 1 Processor -- start 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.817+0000 7f18ef296700 1 -- start start 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.818+0000 7f18ef296700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f18e81a0370 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.821+0000 7f18ed032700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18e8104370 0x7f18e819fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44758/0 (socket says 172.21.15.70:44758) 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.821+0000 7f18ed032700 1 -- 172.21.15.70:0/104909221 learned_addr learned my addr 172.21.15.70:0/104909221 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.821+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930894869 0 0) 0x7f18e81a0370 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d40054e0 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2911599093 0 0) 0x7f18d40054e0 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f18e81a0370 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f18e0001250 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2325316566 0 0) 0x7f18e81a0370 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e81a0500 con 0x7f18e8104370 2024-02-10T07:33:28.716 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e81a07b0 con 0x7f18e8104370 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e81a4300 con 0x7f18e8104370 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f18e00014f0 con 0x7f18e8104370 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.822+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f18e00065d0 con 0x7f18e8104370 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.823+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 9) v1 ==== 45910+0+0 (unknown 3539928795 0 0) 0x7f18e0011ab0 con 0x7f18e8104370 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.823+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6800/1990687880 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18e0001250 con 0x7f18d403a110 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.823+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 2669342012 0 0) 0x7f18e0048680 con 0x7f18e8104370 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:22.823+0000 7f18ec831700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6800/1990687880 conn(0x7f18d403a110 legacy=0x7f18d403c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1990687880 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:23.024+0000 7f18ec831700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6800/1990687880 conn(0x7f18d403a110 legacy=0x7f18d403c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1990687880 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:23.425+0000 7f18ec831700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6800/1990687880 conn(0x7f18d403a110 legacy=0x7f18d403c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1990687880 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:24.226+0000 7f18ec831700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6800/1990687880 conn(0x7f18d403a110 legacy=0x7f18d403c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1990687880 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:25.827+0000 7f18ec831700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6800/1990687880 conn(0x7f18d403a110 legacy=0x7f18d403c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/1990687880 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:26.792+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mgrmap(e 10) v1 ==== 45712+0+0 (unknown 208833788 0 0) 0x7f18e0012fe0 con 0x7f18e8104370 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:26.792+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6800/1990687880 conn(0x7f18d403a110 legacy=0x7f18d403c5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T07:33:28.717 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.793+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mgrmap(e 11) v1 ==== 45804+0+0 (unknown 1864478446 0 0) 0x7f18e0014fc0 con 0x7f18e8104370 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.794+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6800/3858935040 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18e0001250 con 0x7f18d403eb00 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.794+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (unknown 0 0 4109177037) 0x7f18e0001250 con 0x7f18d403eb00 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.801+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 --> v1:172.21.15.70:6800/3858935040 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f18cc002630 con 0x7f18d403eb00 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.801+0000 7f18de7fc700 1 -- 172.21.15.70:0/104909221 <== mgr.14152 v1:172.21.15.70:6800/3858935040 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (unknown 0 0 3086460295) 0x7f18cc002630 con 0x7f18d403eb00 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.802+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6800/3858935040 conn(0x7f18d403eb00 legacy=0x7f18d4040ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.802+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 >> v1:172.21.15.70:6789/0 conn(0x7f18e8104370 legacy=0x7f18e819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.802+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 shutdown_connections 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.802+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 >> 172.21.15.70:0/104909221 conn(0x7f18e80fffd0 msgr2=0x7f18e8106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.802+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 shutdown_connections 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:27.802+0000 7f18ef296700 1 -- 172.21.15.70:0/104909221 wait complete. 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 9 is available 2024-02-10T07:33:28.718 INFO:teuthology.orchestra.run.smithi070.stdout:Generating a dashboard self-signed certificate... 2024-02-10T07:33:28.967 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:27] ENGINE Bus STARTING 2024-02-10T07:33:28.967 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:27] ENGINE Serving on https://172.21.15.70:7150 2024-02-10T07:33:28.967 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: from='client.14156 v1:172.21.15.70:0/104909221' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T07:33:28.967 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: mgrmap e11: y(active, since 1.00563s) 2024-02-10T07:33:28.968 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: from='client.14156 v1:172.21.15.70:0/104909221' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T07:33:28.968 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:27] ENGINE Serving on http://172.21.15.70:8765 2024-02-10T07:33:28.968 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: [10/Feb/2024:07:33:27] ENGINE Bus STARTED 2024-02-10T07:33:28.968 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.192+0000 7f002f8ca700 1 Processor -- start 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.192+0000 7f002f8ca700 1 -- start start 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.192+0000 7f002f8ca700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0028078ed0 con 0x7f0028106520 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f002d666700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0028106520 0x7f0028108910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55768/0 (socket says 172.21.15.70:55768) 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f002d666700 1 -- 172.21.15.70:0/2515861364 learned_addr learned my addr 172.21.15.70:0/2515861364 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f001ffff700 1 -- 172.21.15.70:0/2515861364 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928837364 0 0) 0x7f0028078ed0 con 0x7f0028106520 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f001ffff700 1 -- 172.21.15.70:0/2515861364 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00140035b0 con 0x7f0028106520 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f001ffff700 1 -- 172.21.15.70:0/2515861364 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104416871 0 0) 0x7f00140035b0 con 0x7f0028106520 2024-02-10T07:33:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f001ffff700 1 -- 172.21.15.70:0/2515861364 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002810b040 con 0x7f0028106520 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f001ffff700 1 -- 172.21.15.70:0/2515861364 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f0018001580 con 0x7f0028106520 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f001ffff700 1 -- 172.21.15.70:0/2515861364 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f0018001d00 con 0x7f0028106520 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.193+0000 7f001ffff700 1 -- 172.21.15.70:0/2515861364 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f0018005960 con 0x7f0028106520 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.194+0000 7f002f8ca700 1 -- 172.21.15.70:0/2515861364 >> v1:172.21.15.70:6789/0 conn(0x7f0028106520 legacy=0x7f0028108910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.194+0000 7f002f8ca700 1 -- 172.21.15.70:0/2515861364 shutdown_connections 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.194+0000 7f002f8ca700 1 -- 172.21.15.70:0/2515861364 >> 172.21.15.70:0/2515861364 conn(0x7f00280fff70 msgr2=0x7f0028102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.194+0000 7f002f8ca700 1 -- 172.21.15.70:0/2515861364 shutdown_connections 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.194+0000 7f002f8ca700 1 -- 172.21.15.70:0/2515861364 wait complete. 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.195+0000 7f002f8ca700 1 Processor -- start 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.195+0000 7f002f8ca700 1 -- start start 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.195+0000 7f002f8ca700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0028199cb0 con 0x7f0028106520 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.195+0000 7f002d666700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0028106520 0x7f00281995a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55782/0 (socket says 172.21.15.70:55782) 2024-02-10T07:33:30.104 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.195+0000 7f002d666700 1 -- 172.21.15.70:0/1118483534 learned_addr learned my addr 172.21.15.70:0/1118483534 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.195+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148485435 0 0) 0x7f0028199cb0 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.195+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00140054f0 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 151302070 0 0) 0x7f00140054f0 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0028199cb0 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f0018001080 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1850632854 0 0) 0x7f0028199cb0 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028199e40 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002819a0f0 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f002819dc40 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f0018004a60 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.196+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f0018006140 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.197+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 11) v1 ==== 45804+0+0 (unknown 1864478446 0 0) 0x7f0018011620 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.197+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c005120 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.197+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7f0018047120 con 0x7f0028106520 2024-02-10T07:33:30.105 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.202+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0018012ee0 con 0x7f0028106520 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.402+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f000c000b90 con 0x7f001403a040 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.530+0000 7f001e7fc700 1 -- 172.21.15.70:0/1118483534 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3317752739) 0x7f000c000b90 con 0x7f001403a040 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.533+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 >> v1:172.21.15.70:6800/3858935040 conn(0x7f001403a040 legacy=0x7f001403c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.533+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 >> v1:172.21.15.70:6789/0 conn(0x7f0028106520 legacy=0x7f00281995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.533+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 shutdown_connections 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.533+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 >> 172.21.15.70:0/1118483534 conn(0x7f00280fff70 msgr2=0x7f0028190300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.533+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 shutdown_connections 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:29.533+0000 7f002f8ca700 1 -- 172.21.15.70:0/1118483534 wait complete. 2024-02-10T07:33:30.106 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial admin user... 2024-02-10T07:33:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:30 smithi070 ceph-mon[29548]: from='client.14164 v1:172.21.15.70:0/1118483534' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:30 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:30 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:30 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:30 smithi070 ceph-mon[29548]: mgrmap e12: y(active, since 3s) 2024-02-10T07:33:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:31 smithi070 ceph-mon[29548]: from='client.14166 v1:172.21.15.70:0/2955521897' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:32.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$2AGmxiPrZddaGG8WNF3xz.M7Ksnd7u5hAOvu8h07NMw6yW/x0vv9y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707550410, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-10T07:33:32.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.507+0000 7f21b03bc700 1 Processor -- start 2024-02-10T07:33:32.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.507+0000 7f21b03bc700 1 -- start start 2024-02-10T07:33:32.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21b03bc700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21a8109f60 con 0x7f21a8103da0 2024-02-10T07:33:32.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21ae158700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f21a8103da0 0x7f21a81041c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55796/0 (socket says 172.21.15.70:55796) 2024-02-10T07:33:32.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21ae158700 1 -- 172.21.15.70:0/1053654501 learned_addr learned my addr 172.21.15.70:0/1053654501 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:32.065 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21ad156700 1 -- 172.21.15.70:0/1053654501 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520264261 0 0) 0x7f21a8109f60 con 0x7f21a8103da0 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21ad156700 1 -- 172.21.15.70:0/1053654501 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21940035b0 con 0x7f21a8103da0 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21ad156700 1 -- 172.21.15.70:0/1053654501 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2098638246 0 0) 0x7f21940035b0 con 0x7f21a8103da0 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21ad156700 1 -- 172.21.15.70:0/1053654501 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a810b100 con 0x7f21a8103da0 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.508+0000 7f21ad156700 1 -- 172.21.15.70:0/1053654501 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f2198001580 con 0x7f21a8103da0 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.509+0000 7f21ad156700 1 -- 172.21.15.70:0/1053654501 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f2198001d00 con 0x7f21a8103da0 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.509+0000 7f21ad156700 1 -- 172.21.15.70:0/1053654501 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f2198005960 con 0x7f21a8103da0 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.509+0000 7f21b03bc700 1 -- 172.21.15.70:0/1053654501 >> v1:172.21.15.70:6789/0 conn(0x7f21a8103da0 legacy=0x7f21a81041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.509+0000 7f21b03bc700 1 -- 172.21.15.70:0/1053654501 shutdown_connections 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.509+0000 7f21b03bc700 1 -- 172.21.15.70:0/1053654501 >> 172.21.15.70:0/1053654501 conn(0x7f21a80ff6f0 msgr2=0x7f21a8101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.510+0000 7f21b03bc700 1 -- 172.21.15.70:0/1053654501 shutdown_connections 2024-02-10T07:33:32.066 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.510+0000 7f21b03bc700 1 -- 172.21.15.70:0/1053654501 wait complete. 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.510+0000 7f21b03bc700 1 Processor -- start 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.510+0000 7f21b03bc700 1 -- start start 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.510+0000 7f21b03bc700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21a8199c90 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f21ae158700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f21a8103da0 0x7f21a8199580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55808/0 (socket says 172.21.15.70:55808) 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f21ae158700 1 -- 172.21.15.70:0/2955521897 learned_addr learned my addr 172.21.15.70:0/2955521897 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375824273 0 0) 0x7f21a8199c90 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21940054f0 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1267994368 0 0) 0x7f21940054f0 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21a8199c90 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f2198001080 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4199581959 0 0) 0x7f21a8199c90 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8199e20 con 0x7f21a8103da0 2024-02-10T07:33:32.067 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a819a0d0 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.511+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a819dc20 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.512+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f2198004a60 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.512+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f21980060d0 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.512+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218c005150 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.512+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 12) v1 ==== 45910+0+0 (unknown 3707513077 0 0) 0x7f21980115b0 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.513+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7f21980472d0 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.518+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f2198012e60 con 0x7f21a8103da0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.739+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f218c002430 con 0x7f219403a1b0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.965+0000 7f219f7fe700 1 -- 172.21.15.70:0/2955521897 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (unknown 0 0 2716883831) 0x7f218c002430 con 0x7f219403a1b0 2024-02-10T07:33:32.068 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.967+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 >> v1:172.21.15.70:6800/3858935040 conn(0x7f219403a1b0 legacy=0x7f219403c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.967+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 >> v1:172.21.15.70:6789/0 conn(0x7f21a8103da0 legacy=0x7f21a8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.967+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 shutdown_connections 2024-02-10T07:33:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.967+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 >> 172.21.15.70:0/2955521897 conn(0x7f21a80ff6f0 msgr2=0x7f21a8190330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.967+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 shutdown_connections 2024-02-10T07:33:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:30.967+0000 7f21b03bc700 1 -- 172.21.15.70:0/2955521897 wait complete. 2024-02-10T07:33:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:Fetching dashboard port number... 2024-02-10T07:33:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2653924439' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 8443 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.467+0000 7f25c8f1d700 1 Processor -- start 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.467+0000 7f25c8f1d700 1 -- start start 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.467+0000 7f25c8f1d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f25c4078e10 con 0x7f25c41065e0 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c259c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f25c41065e0 0x7f25c41089d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55818/0 (socket says 172.21.15.70:55818) 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c259c700 1 -- 172.21.15.70:0/823497500 learned_addr learned my addr 172.21.15.70:0/823497500 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c1d9b700 1 -- 172.21.15.70:0/823497500 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241142579 0 0) 0x7f25c4078e10 con 0x7f25c41065e0 2024-02-10T07:33:33.306 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c1d9b700 1 -- 172.21.15.70:0/823497500 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b00035b0 con 0x7f25c41065e0 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c1d9b700 1 -- 172.21.15.70:0/823497500 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 133966591 0 0) 0x7f25b00035b0 con 0x7f25c41065e0 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c1d9b700 1 -- 172.21.15.70:0/823497500 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c410b100 con 0x7f25c41065e0 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c1d9b700 1 -- 172.21.15.70:0/823497500 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f25b4001900 con 0x7f25c41065e0 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c1d9b700 1 -- 172.21.15.70:0/823497500 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f25b4002080 con 0x7f25c41065e0 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.468+0000 7f25c1d9b700 1 -- 172.21.15.70:0/823497500 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f25b4005ce0 con 0x7f25c41065e0 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.469+0000 7f25c8f1d700 1 -- 172.21.15.70:0/823497500 >> v1:172.21.15.70:6789/0 conn(0x7f25c41065e0 legacy=0x7f25c41089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.469+0000 7f25c8f1d700 1 -- 172.21.15.70:0/823497500 shutdown_connections 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.469+0000 7f25c8f1d700 1 -- 172.21.15.70:0/823497500 >> 172.21.15.70:0/823497500 conn(0x7f25c40fff70 msgr2=0x7f25c4102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.469+0000 7f25c8f1d700 1 -- 172.21.15.70:0/823497500 shutdown_connections 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.469+0000 7f25c8f1d700 1 -- 172.21.15.70:0/823497500 wait complete. 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.469+0000 7f25c8f1d700 1 Processor -- start 2024-02-10T07:33:33.307 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25c8f1d700 1 -- start start 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25c8f1d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f25c419f5f0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25c259c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f25c41065e0 0x7f25c4105e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55820/0 (socket says 172.21.15.70:55820) 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25c259c700 1 -- 172.21.15.70:0/2653924439 learned_addr learned my addr 172.21.15.70:0/2653924439 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672199880 0 0) 0x7f25c419f5f0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b00054f0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2229555769 0 0) 0x7f25b00054f0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f25c419f5f0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f25b4001400 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 832826732 0 0) 0x7f25c419f5f0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.470+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c41a0790 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.471+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c419f7e0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.471+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c419fce0 con 0x7f25c41065e0 2024-02-10T07:33:33.308 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.471+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f25b4001bf0 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.471+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f25b4006540 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.471+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 12) v1 ==== 45910+0+0 (unknown 3707513077 0 0) 0x7f25b4011a00 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.471+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a4005120 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.472+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7f25b4047b10 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.476+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f25b4013300 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.669+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f25a4005950 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.670+0000 7f25bbfff700 1 -- 172.21.15.70:0/2653924439 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7f25b40153e0 con 0x7f25c41065e0 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.672+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 >> v1:172.21.15.70:6800/3858935040 conn(0x7f25b003a190 legacy=0x7f25b003c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.672+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 >> v1:172.21.15.70:6789/0 conn(0x7f25c41065e0 legacy=0x7f25c4105e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.672+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 shutdown_connections 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.672+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 >> 172.21.15.70:0/2653924439 conn(0x7f25c40fff70 msgr2=0x7f25c4190330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.672+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 shutdown_connections 2024-02-10T07:33:33.309 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:32.672+0000 7f25c8f1d700 1 -- 172.21.15.70:0/2653924439 wait complete. 2024-02-10T07:33:33.313 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:33:33.313 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-02-10T07:33:33.321 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:33:33.321 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-02-10T07:33:33.321 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-02-10T07:33:33.321 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-10T07:33:33.322 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph Dashboard is now available at: 2024-02-10T07:33:33.322 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:33.322 INFO:teuthology.orchestra.run.smithi070.stdout: URL: https://smithi070.front.sepia.ceph.com:8443/ 2024-02-10T07:33:33.322 INFO:teuthology.orchestra.run.smithi070.stdout: User: admin 2024-02-10T07:33:33.323 INFO:teuthology.orchestra.run.smithi070.stdout: Password: rw3gkq8p9n 2024-02-10T07:33:33.323 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:33.323 INFO:teuthology.orchestra.run.smithi070.stdout:Saving cluster configuration to /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config directory 2024-02-10T07:33:33.323 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling autotune for osd_memory_target 2024-02-10T07:33:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:33 smithi070 ceph-mon[29548]: mgrmap e13: y(active, since 6s) 2024-02-10T07:33:34.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.753+0000 7fa73d39c700 1 Processor -- start 2024-02-10T07:33:34.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.753+0000 7fa73d39c700 1 -- start start 2024-02-10T07:33:34.454 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.753+0000 7fa73d39c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa738109f30 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa736ffd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa738105b00 0x7fa738105f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55826/0 (socket says 172.21.15.70:55826) 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa736ffd700 1 -- 172.21.15.70:0/2159866609 learned_addr learned my addr 172.21.15.70:0/2159866609 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa7367fc700 1 -- 172.21.15.70:0/2159866609 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322049783 0 0) 0x7fa738109f30 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa7367fc700 1 -- 172.21.15.70:0/2159866609 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7240035b0 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa7367fc700 1 -- 172.21.15.70:0/2159866609 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3317245448 0 0) 0x7fa7240035b0 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa7367fc700 1 -- 172.21.15.70:0/2159866609 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73810b0d0 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa7367fc700 1 -- 172.21.15.70:0/2159866609 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa728001900 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa7367fc700 1 -- 172.21.15.70:0/2159866609 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fa728001fb0 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.754+0000 7fa7367fc700 1 -- 172.21.15.70:0/2159866609 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa728005af0 con 0x7fa738105b00 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.755+0000 7fa73d39c700 1 -- 172.21.15.70:0/2159866609 >> v1:172.21.15.70:6789/0 conn(0x7fa738105b00 legacy=0x7fa738105f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.755+0000 7fa73d39c700 1 -- 172.21.15.70:0/2159866609 shutdown_connections 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.755+0000 7fa73d39c700 1 -- 172.21.15.70:0/2159866609 >> 172.21.15.70:0/2159866609 conn(0x7fa7381010e0 msgr2=0x7fa738103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:34.455 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.755+0000 7fa73d39c700 1 -- 172.21.15.70:0/2159866609 shutdown_connections 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.755+0000 7fa73d39c700 1 -- 172.21.15.70:0/2159866609 wait complete. 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.755+0000 7fa73d39c700 1 Processor -- start 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa73d39c700 1 -- start start 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa73d39c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa738199ca0 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa736ffd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa738105b00 0x7fa738199590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55830/0 (socket says 172.21.15.70:55830) 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa736ffd700 1 -- 172.21.15.70:0/3656094356 learned_addr learned my addr 172.21.15.70:0/3656094356 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062051820 0 0) 0x7fa738199ca0 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa724005540 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 135257735 0 0) 0x7fa724005540 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa738199ca0 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.756+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa728001400 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 62316213 0 0) 0x7fa738199ca0 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738199e30 con 0x7fa738105b00 2024-02-10T07:33:34.456 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73819a0e0 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73819dc30 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fa728004c80 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fa728006570 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7fa728011b40 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.757+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa718005150 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.758+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7fa7280479f0 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.762+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fa7280134b0 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.957+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 --> v1:172.21.15.70:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa718005980 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.958+0000 7fa734ff9700 1 -- 172.21.15.70:0/3656094356 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (unknown 689008156 0 0) 0x7fa728015590 con 0x7fa738105b00 2024-02-10T07:33:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.960+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 >> v1:172.21.15.70:6800/3858935040 conn(0x7fa72403a160 legacy=0x7fa72403c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:34.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.960+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 >> v1:172.21.15.70:6789/0 conn(0x7fa738105b00 legacy=0x7fa738199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:34.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.960+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 shutdown_connections 2024-02-10T07:33:34.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.960+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 >> 172.21.15.70:0/3656094356 conn(0x7fa7381010e0 msgr2=0x7fa738190360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:34.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.960+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 shutdown_connections 2024-02-10T07:33:34.458 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:33.960+0000 7fa73d39c700 1 -- 172.21.15.70:0/3656094356 wait complete. 2024-02-10T07:33:35.683 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.875+0000 7f726f082700 1 Processor -- start 2024-02-10T07:33:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.875+0000 7f726f082700 1 -- start start 2024-02-10T07:33:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.875+0000 7f726f082700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7268109fe0 con 0x7f7268103e10 2024-02-10T07:33:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.875+0000 7f726ce1e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7268103e10 0x7f7268104230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55842/0 (socket says 172.21.15.70:55842) 2024-02-10T07:33:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.875+0000 7f726ce1e700 1 -- 172.21.15.70:0/3205462096 learned_addr learned my addr 172.21.15.70:0/3205462096 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.875+0000 7f72677fe700 1 -- 172.21.15.70:0/3205462096 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040720437 0 0) 0x7f7268109fe0 con 0x7f7268103e10 2024-02-10T07:33:35.684 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f72677fe700 1 -- 172.21.15.70:0/3205462096 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72580035b0 con 0x7f7268103e10 2024-02-10T07:33:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f72677fe700 1 -- 172.21.15.70:0/3205462096 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3218160929 0 0) 0x7f72580035b0 con 0x7f7268103e10 2024-02-10T07:33:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f72677fe700 1 -- 172.21.15.70:0/3205462096 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726810b180 con 0x7f7268103e10 2024-02-10T07:33:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f72677fe700 1 -- 172.21.15.70:0/3205462096 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f725c001580 con 0x7f7268103e10 2024-02-10T07:33:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f72677fe700 1 -- 172.21.15.70:0/3205462096 <== mon.0 v1:172.21.15.70:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f725c001d00 con 0x7f7268103e10 2024-02-10T07:33:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f72677fe700 1 -- 172.21.15.70:0/3205462096 <== mon.0 v1:172.21.15.70:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f725c005960 con 0x7f7268103e10 2024-02-10T07:33:35.685 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f726f082700 1 -- 172.21.15.70:0/3205462096 >> v1:172.21.15.70:6789/0 conn(0x7f7268103e10 legacy=0x7f7268104230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:35.686 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f726f082700 1 -- 172.21.15.70:0/3205462096 shutdown_connections 2024-02-10T07:33:35.686 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f726f082700 1 -- 172.21.15.70:0/3205462096 >> 172.21.15.70:0/3205462096 conn(0x7f72680ff6f0 msgr2=0x7f7268101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:35.686 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f726f082700 1 -- 172.21.15.70:0/3205462096 shutdown_connections 2024-02-10T07:33:35.686 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.876+0000 7f726f082700 1 -- 172.21.15.70:0/3205462096 wait complete. 2024-02-10T07:33:35.686 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.877+0000 7f726f082700 1 Processor -- start 2024-02-10T07:33:35.686 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.877+0000 7f726f082700 1 -- start start 2024-02-10T07:33:35.687 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.877+0000 7f726f082700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7268199cb0 con 0x7f7268103e10 2024-02-10T07:33:35.687 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.877+0000 7f726ce1e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7268103e10 0x7f72681995a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55852/0 (socket says 172.21.15.70:55852) 2024-02-10T07:33:35.687 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.877+0000 7f726ce1e700 1 -- 172.21.15.70:0/2788324119 learned_addr learned my addr 172.21.15.70:0/2788324119 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:35.687 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801579879 0 0) 0x7f7268199cb0 con 0x7f7268103e10 2024-02-10T07:33:35.687 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72580054f0 con 0x7f7268103e10 2024-02-10T07:33:35.687 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2004533999 0 0) 0x7f72580054f0 con 0x7f7268103e10 2024-02-10T07:33:35.688 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7268199cb0 con 0x7f7268103e10 2024-02-10T07:33:35.688 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f725c001080 con 0x7f7268103e10 2024-02-10T07:33:35.688 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3201160846 0 0) 0x7f7268199cb0 con 0x7f7268103e10 2024-02-10T07:33:35.688 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268199e40 con 0x7f7268103e10 2024-02-10T07:33:35.688 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726819a0f0 con 0x7f7268103e10 2024-02-10T07:33:35.688 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f725c004a60 con 0x7f7268103e10 2024-02-10T07:33:35.689 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f725c006220 con 0x7f7268103e10 2024-02-10T07:33:35.689 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.878+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f726819dc40 con 0x7f7268103e10 2024-02-10T07:33:35.689 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.879+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7f725c011880 con 0x7f7268103e10 2024-02-10T07:33:35.689 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.879+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7f725c048390 con 0x7f7268103e10 2024-02-10T07:33:35.689 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.879+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7268193270 con 0x7f7268103e10 2024-02-10T07:33:35.689 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:34.884+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f725c016d40 con 0x7f7268103e10 2024-02-10T07:33:35.689 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.184+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 --> v1:172.21.15.70:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f726810b310 con 0x7f7268103e10 2024-02-10T07:33:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.187+0000 7f7265ffb700 1 -- 172.21.15.70:0/2788324119 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (unknown 1169358022 0 0) 0x7f725c047c80 con 0x7f7268103e10 2024-02-10T07:33:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.189+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 >> v1:172.21.15.70:6800/3858935040 conn(0x7f725803a210 legacy=0x7f725803c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.189+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 >> v1:172.21.15.70:6789/0 conn(0x7f7268103e10 legacy=0x7f72681995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.189+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 shutdown_connections 2024-02-10T07:33:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.189+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 >> 172.21.15.70:0/2788324119 conn(0x7f72680ff6f0 msgr2=0x7f72681903b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:35.690 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.189+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 shutdown_connections 2024-02-10T07:33:35.691 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr 2024-02-10T07:33:35.189+0000 7f726f082700 1 -- 172.21.15.70:0/2788324119 wait complete. 2024-02-10T07:33:35.691 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-10T07:33:35.691 INFO:teuthology.orchestra.run.smithi070.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-10T07:33:35.691 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.691 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:33:35.691 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.691 INFO:teuthology.orchestra.run.smithi070.stdout:Or, if you are only running a single cluster on this host: 2024-02-10T07:33:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-10T07:33:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.692 INFO:teuthology.orchestra.run.smithi070.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-10T07:33:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.692 INFO:teuthology.orchestra.run.smithi070.stdout: ceph telemetry on 2024-02-10T07:33:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.693 INFO:teuthology.orchestra.run.smithi070.stdout:For more information see: 2024-02-10T07:33:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-10T07:33:35.693 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:35.693 INFO:teuthology.orchestra.run.smithi070.stdout:Bootstrap complete. 2024-02-10T07:33:35.717 INFO:tasks.cephadm:Fetching config... 2024-02-10T07:33:35.718 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:33:35.718 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-10T07:33:35.733 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-10T07:33:35.734 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:33:35.734 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-10T07:33:35.789 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-10T07:33:35.789 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:33:35.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-10T07:33:35.855 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-10T07:33:35.856 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:33:35.856 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-10T07:33:35.911 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-10T07:33:35.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCFv4C86/2hWBdnb2iRbuWscd4ji35DL3uSGDYPlogdigDJid7acqEI6Z/GFfxvYF270nd0Ol6yLRfktqzz/1squKPCzIg8R0Ms1Rqodpu01sjCzjNWk0YVJRYxpE1K1PN02QPA52xJdL/c6KiAqLkHOL/bZ3PDlngiShuiiy7/yeP0yOyWGKrSeXup2m5EbsQwGT/os4bbDS4KWX81mhL6QpkSsHV+ebu65pqoTTng8KyCYW6BmdxvSErU1GsXc3mGpmbN1dICgt/IY+eoPf0jlFBwJrXRMKayoz8wzPyh9h1Qlrl8j/NMXufsDunhPnXv0ZSz6vAy9VpwXxq/F5w4Blht5PsIMC8rUB29RaLPNC0OcDZc4JqWql1/9md40SOzAFzkDVURK+xJeseohDXGuYdC9lORIkc4mN7ZqXtLL3mBKDqEajziHgvWk9tIEraXPPcFaPqzvq1pZ99z6N7vfEBQ3u1iPHmdCetCD9iBFI0ZaD57cdcPqjDuP3slO0= ceph-70142e68-c7e6-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T07:33:35.989 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCFv4C86/2hWBdnb2iRbuWscd4ji35DL3uSGDYPlogdigDJid7acqEI6Z/GFfxvYF270nd0Ol6yLRfktqzz/1squKPCzIg8R0Ms1Rqodpu01sjCzjNWk0YVJRYxpE1K1PN02QPA52xJdL/c6KiAqLkHOL/bZ3PDlngiShuiiy7/yeP0yOyWGKrSeXup2m5EbsQwGT/os4bbDS4KWX81mhL6QpkSsHV+ebu65pqoTTng8KyCYW6BmdxvSErU1GsXc3mGpmbN1dICgt/IY+eoPf0jlFBwJrXRMKayoz8wzPyh9h1Qlrl8j/NMXufsDunhPnXv0ZSz6vAy9VpwXxq/F5w4Blht5PsIMC8rUB29RaLPNC0OcDZc4JqWql1/9md40SOzAFzkDVURK+xJeseohDXGuYdC9lORIkc4mN7ZqXtLL3mBKDqEajziHgvWk9tIEraXPPcFaPqzvq1pZ99z6N7vfEBQ3u1iPHmdCetCD9iBFI0ZaD57cdcPqjDuP3slO0= ceph-70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:33:36.003 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCFv4C86/2hWBdnb2iRbuWscd4ji35DL3uSGDYPlogdigDJid7acqEI6Z/GFfxvYF270nd0Ol6yLRfktqzz/1squKPCzIg8R0Ms1Rqodpu01sjCzjNWk0YVJRYxpE1K1PN02QPA52xJdL/c6KiAqLkHOL/bZ3PDlngiShuiiy7/yeP0yOyWGKrSeXup2m5EbsQwGT/os4bbDS4KWX81mhL6QpkSsHV+ebu65pqoTTng8KyCYW6BmdxvSErU1GsXc3mGpmbN1dICgt/IY+eoPf0jlFBwJrXRMKayoz8wzPyh9h1Qlrl8j/NMXufsDunhPnXv0ZSz6vAy9VpwXxq/F5w4Blht5PsIMC8rUB29RaLPNC0OcDZc4JqWql1/9md40SOzAFzkDVURK+xJeseohDXGuYdC9lORIkc4mN7ZqXtLL3mBKDqEajziHgvWk9tIEraXPPcFaPqzvq1pZ99z6N7vfEBQ3u1iPHmdCetCD9iBFI0ZaD57cdcPqjDuP3slO0= ceph-70142e68-c7e6-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T07:33:36.049 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCFv4C86/2hWBdnb2iRbuWscd4ji35DL3uSGDYPlogdigDJid7acqEI6Z/GFfxvYF270nd0Ol6yLRfktqzz/1squKPCzIg8R0Ms1Rqodpu01sjCzjNWk0YVJRYxpE1K1PN02QPA52xJdL/c6KiAqLkHOL/bZ3PDlngiShuiiy7/yeP0yOyWGKrSeXup2m5EbsQwGT/os4bbDS4KWX81mhL6QpkSsHV+ebu65pqoTTng8KyCYW6BmdxvSErU1GsXc3mGpmbN1dICgt/IY+eoPf0jlFBwJrXRMKayoz8wzPyh9h1Qlrl8j/NMXufsDunhPnXv0ZSz6vAy9VpwXxq/F5w4Blht5PsIMC8rUB29RaLPNC0OcDZc4JqWql1/9md40SOzAFzkDVURK+xJeseohDXGuYdC9lORIkc4mN7ZqXtLL3mBKDqEajziHgvWk9tIEraXPPcFaPqzvq1pZ99z6N7vfEBQ3u1iPHmdCetCD9iBFI0ZaD57cdcPqjDuP3slO0= ceph-70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:33:36.064 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-10T07:33:36.386 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:33:36.483 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2788324119' entity='client.admin' 2024-02-10T07:33:38.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:38.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:38.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:33:38.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:38.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:38.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.895+0000 7fbf0608f700 1 -- 172.21.15.70:0/3098366022 >> v1:172.21.15.70:6789/0 conn(0x7fbf00102100 legacy=0x7fbf001044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.895+0000 7fbf0608f700 1 -- 172.21.15.70:0/3098366022 shutdown_connections 2024-02-10T07:33:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.895+0000 7fbf0608f700 1 -- 172.21.15.70:0/3098366022 >> 172.21.15.70:0/3098366022 conn(0x7fbf000fbb10 msgr2=0x7fbf000fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.895+0000 7fbf0608f700 1 -- 172.21.15.70:0/3098366022 shutdown_connections 2024-02-10T07:33:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.895+0000 7fbf0608f700 1 -- 172.21.15.70:0/3098366022 wait complete. 2024-02-10T07:33:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.896+0000 7fbf0608f700 1 Processor -- start 2024-02-10T07:33:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.896+0000 7fbf0608f700 1 -- start start 2024-02-10T07:33:38.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.896+0000 7fbf0608f700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbf00196630 con 0x7fbf00102100 2024-02-10T07:33:38.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbeff7fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbf00102100 0x7fbf00195f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53688/0 (socket says 172.21.15.70:53688) 2024-02-10T07:33:38.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbeff7fe700 1 -- 172.21.15.70:0/2238142148 learned_addr learned my addr 172.21.15.70:0/2238142148 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:38.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51845793 0 0) 0x7fbf00196630 con 0x7fbf00102100 2024-02-10T07:33:38.898 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee80054f0 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2471642745 0 0) 0x7fbee80054f0 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf00196630 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbef00044b0 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3149278973 0 0) 0x7fbf00196630 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf001967c0 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf00196a70 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.897+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0019a5c0 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.898+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fbef0004700 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.898+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fbef0004f00 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.898+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7fbef0005080 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.899+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeec005150 con 0x7fbf00102100 2024-02-10T07:33:38.899 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.899+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7fbef0048240 con 0x7fbf00102100 2024-02-10T07:33:38.904 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:38.904+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fbef0005470 con 0x7fbf00102100 2024-02-10T07:33:39.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.102+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 --> v1:172.21.15.70:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fbeec005980 con 0x7fbf00102100 2024-02-10T07:33:39.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.105+0000 7fbefcff9700 1 -- 172.21.15.70:0/2238142148 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (unknown 3028693289 0 0) 0x7fbef0014fc0 con 0x7fbf00102100 2024-02-10T07:33:39.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.107+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 >> v1:172.21.15.70:6800/3858935040 conn(0x7fbee803a250 legacy=0x7fbee803c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:39.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.107+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 >> v1:172.21.15.70:6789/0 conn(0x7fbf00102100 legacy=0x7fbf00195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:39.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.107+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 shutdown_connections 2024-02-10T07:33:39.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.107+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 >> 172.21.15.70:0/2238142148 conn(0x7fbf000fbb10 msgr2=0x7fbf00104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:39.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.107+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 shutdown_connections 2024-02-10T07:33:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:39.107+0000 7fbf0608f700 1 -- 172.21.15.70:0/2238142148 wait complete. 2024-02-10T07:33:39.839 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-10T07:33:39.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-10T07:33:40.306 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:33:40.424 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2238142148' entity='client.admin' 2024-02-10T07:33:40.424 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:40 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:40.424 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:40 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.867+0000 7f8a58fb2700 1 -- 172.21.15.70:0/24178804 >> v1:172.21.15.70:6789/0 conn(0x7f8a54101680 legacy=0x7f8a54101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.867+0000 7f8a58fb2700 1 -- 172.21.15.70:0/24178804 shutdown_connections 2024-02-10T07:33:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.867+0000 7f8a58fb2700 1 -- 172.21.15.70:0/24178804 >> 172.21.15.70:0/24178804 conn(0x7f8a540fcc20 msgr2=0x7f8a540ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.867+0000 7f8a58fb2700 1 -- 172.21.15.70:0/24178804 shutdown_connections 2024-02-10T07:33:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.867+0000 7f8a58fb2700 1 -- 172.21.15.70:0/24178804 wait complete. 2024-02-10T07:33:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.868+0000 7f8a58fb2700 1 Processor -- start 2024-02-10T07:33:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.868+0000 7f8a58fb2700 1 -- start start 2024-02-10T07:33:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.868+0000 7f8a58fb2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8a54196620 con 0x7f8a54101680 2024-02-10T07:33:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.868+0000 7f8a5259c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8a54101680 0x7f8a54195f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53700/0 (socket says 172.21.15.70:53700) 2024-02-10T07:33:41.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.868+0000 7f8a5259c700 1 -- 172.21.15.70:0/3024163887 learned_addr learned my addr 172.21.15.70:0/3024163887 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.868+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043995370 0 0) 0x7f8a54196620 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.868+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a3c0054f0 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1153002782 0 0) 0x7f8a3c0054f0 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8a54196620 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f8a48004440 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2876532205 0 0) 0x7f8a54196620 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a541967b0 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a54196a60 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5419a560 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f8a48001620 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.869+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f8a48004eb0 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.870+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a34005150 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.870+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7f8a48005030 con 0x7f8a54101680 2024-02-10T07:33:41.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.870+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7f8a480474f0 con 0x7f8a54101680 2024-02-10T07:33:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:41.875+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8a48005420 con 0x7f8a54101680 2024-02-10T07:33:42.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.070+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a34000bc0 con 0x7f8a3c03a240 2024-02-10T07:33:42.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.073+0000 7f8a437fe700 1 -- 172.21.15.70:0/3024163887 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f8a34000bc0 con 0x7f8a3c03a240 2024-02-10T07:33:42.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.075+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 >> v1:172.21.15.70:6800/3858935040 conn(0x7f8a3c03a240 legacy=0x7f8a3c03c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:42.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.075+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 >> v1:172.21.15.70:6789/0 conn(0x7f8a54101680 legacy=0x7f8a54195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:42.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.075+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 shutdown_connections 2024-02-10T07:33:42.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.075+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 >> 172.21.15.70:0/3024163887 conn(0x7f8a540fcc20 msgr2=0x7f8a540ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:42.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.075+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 shutdown_connections 2024-02-10T07:33:42.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:42.075+0000 7f8a58fb2700 1 -- 172.21.15.70:0/3024163887 wait complete. 2024-02-10T07:33:42.785 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-02-10T07:33:42.785 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:33:42.785 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-02-10T07:33:42.809 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:33:42.809 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:33:42.868 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-02-10T07:33:42.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch host add smithi155 2024-02-10T07:33:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='client.14176 v1:172.21.15.70:0/3024163887' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: Updating smithi070:/etc/ceph/ceph.conf 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:33:43.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:43.332 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:43.332 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:44.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.519+0000 7fb618e6a700 1 -- 172.21.15.70:0/358799186 >> v1:172.21.15.70:6789/0 conn(0x7fb614106690 legacy=0x7fb614106a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:44.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.519+0000 7fb618e6a700 1 -- 172.21.15.70:0/358799186 shutdown_connections 2024-02-10T07:33:44.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.519+0000 7fb618e6a700 1 -- 172.21.15.70:0/358799186 >> 172.21.15.70:0/358799186 conn(0x7fb6140fb7a0 msgr2=0x7fb6140fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:44.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.519+0000 7fb618e6a700 1 -- 172.21.15.70:0/358799186 shutdown_connections 2024-02-10T07:33:44.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.519+0000 7fb618e6a700 1 -- 172.21.15.70:0/358799186 wait complete. 2024-02-10T07:33:44.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.519+0000 7fb618e6a700 1 Processor -- start 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.519+0000 7fb618e6a700 1 -- start start 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb618e6a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb6140fdaf0 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb61259c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb614106690 0x7fb6140fbc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53720/0 (socket says 172.21.15.70:53720) 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb61259c700 1 -- 172.21.15.70:0/3107664127 learned_addr learned my addr 172.21.15.70:0/3107664127 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894961304 0 0) 0x7fb6140fdaf0 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6000054f0 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2810218692 0 0) 0x7fb6000054f0 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6140fdaf0 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fb604001980 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1843196043 0 0) 0x7fb6140fdaf0 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6140fc360 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6140fc610 con 0x7fb614106690 2024-02-10T07:33:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6140fca90 con 0x7fb614106690 2024-02-10T07:33:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fb604001200 con 0x7fb614106690 2024-02-10T07:33:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.520+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fb604004e90 con 0x7fb614106690 2024-02-10T07:33:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.521+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7fb6040050d0 con 0x7fb614106690 2024-02-10T07:33:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.521+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f4005120 con 0x7fb614106690 2024-02-10T07:33:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.521+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7fb604047500 con 0x7fb614106690 2024-02-10T07:33:44.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.526+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fb604013480 con 0x7fb614106690 2024-02-10T07:33:44.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:44.721+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5f4000b90 con 0x7fb60003a1f0 2024-02-10T07:33:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:45 smithi070 ceph-mon[29548]: from='client.14178 v1:172.21.15.70:0/3107664127' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:46.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:46 smithi070 ceph-mon[29548]: Deploying cephadm binary to smithi155 2024-02-10T07:33:48.229 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:47 smithi070 ceph-mon[29548]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:33:49.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:49.086+0000 7fb60b7fe700 1 -- 172.21.15.70:0/3107664127 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 2103545083) 0x7fb5f4000b90 con 0x7fb60003a1f0 2024-02-10T07:33:49.087 INFO:teuthology.orchestra.run.smithi070.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-02-10T07:33:49.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:49.088+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 >> v1:172.21.15.70:6800/3858935040 conn(0x7fb60003a1f0 legacy=0x7fb60003c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:49.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:49.088+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 >> v1:172.21.15.70:6789/0 conn(0x7fb614106690 legacy=0x7fb6140fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:49.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:49.088+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 shutdown_connections 2024-02-10T07:33:49.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:49.088+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 >> 172.21.15.70:0/3107664127 conn(0x7fb6140fb7a0 msgr2=0x7fb6140ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:49.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:49.088+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 shutdown_connections 2024-02-10T07:33:49.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:49.088+0000 7fb618e6a700 1 -- 172.21.15.70:0/3107664127 wait complete. 2024-02-10T07:33:49.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch host ls --format=json 2024-02-10T07:33:49.898 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:33:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:49 smithi070 ceph-mon[29548]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:33:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:50.941 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:50 smithi070 ceph-mon[29548]: Added host smithi155 2024-02-10T07:33:50.941 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:50 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.434+0000 7fc06fb25700 1 -- 172.21.15.70:0/409341367 >> v1:172.21.15.70:6789/0 conn(0x7fc0681065e0 legacy=0x7fc0681069c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.434+0000 7fc06fb25700 1 -- 172.21.15.70:0/409341367 shutdown_connections 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.434+0000 7fc06fb25700 1 -- 172.21.15.70:0/409341367 >> 172.21.15.70:0/409341367 conn(0x7fc0680fbfe0 msgr2=0x7fc0680fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.434+0000 7fc06fb25700 1 -- 172.21.15.70:0/409341367 shutdown_connections 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.434+0000 7fc06fb25700 1 -- 172.21.15.70:0/409341367 wait complete. 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.435+0000 7fc06fb25700 1 Processor -- start 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.435+0000 7fc06fb25700 1 -- start start 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.435+0000 7fc06fb25700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc068197770 con 0x7fc0681065e0 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.435+0000 7fc06d8c1700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc0681065e0 0x7fc068197060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55666/0 (socket says 172.21.15.70:55666) 2024-02-10T07:33:51.438 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.435+0000 7fc06d8c1700 1 -- 172.21.15.70:0/1549975816 learned_addr learned my addr 172.21.15.70:0/1549975816 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.435+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841916923 0 0) 0x7fc068197770 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.435+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0540054f0 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 938102956 0 0) 0x7fc0540054f0 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc068197770 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc064001980 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2693718476 0 0) 0x7fc068197770 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068197900 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc068197bb0 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06819f0d0 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fc064001620 con 0x7fc0681065e0 2024-02-10T07:33:51.439 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.436+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc064004e50 con 0x7fc0681065e0 2024-02-10T07:33:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.437+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c005120 con 0x7fc0681065e0 2024-02-10T07:33:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.437+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7fc064005090 con 0x7fc0681065e0 2024-02-10T07:33:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.437+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7fc064047370 con 0x7fc0681065e0 2024-02-10T07:33:51.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.443+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc0640168f0 con 0x7fc0681065e0 2024-02-10T07:33:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.640+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc04c000b90 con 0x7fc05403a240 2024-02-10T07:33:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.641+0000 7fc05effd700 1 -- 172.21.15.70:0/1549975816 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (unknown 0 0 1486324033) 0x7fc04c000b90 con 0x7fc05403a240 2024-02-10T07:33:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:33:51.643 INFO:teuthology.orchestra.run.smithi070.stdout:[{"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-02-10T07:33:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.644+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 >> v1:172.21.15.70:6800/3858935040 conn(0x7fc05403a240 legacy=0x7fc05403c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.644+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 >> v1:172.21.15.70:6789/0 conn(0x7fc0681065e0 legacy=0x7fc068197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.644+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 shutdown_connections 2024-02-10T07:33:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.644+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 >> 172.21.15.70:0/1549975816 conn(0x7fc0680fbfe0 msgr2=0x7fc0681074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.644+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 shutdown_connections 2024-02-10T07:33:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:51.644+0000 7fc06fb25700 1 -- 172.21.15.70:0/1549975816 wait complete. 2024-02-10T07:33:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:51 smithi070 ceph-mon[29548]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:33:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:52.157 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-10T07:33:52.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd crush tunables default 2024-02-10T07:33:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:33:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:52 smithi070 ceph-mon[29548]: from='client.14180 v1:172.21.15.70:0/1549975816' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:33:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.858+0000 7f71e0a21700 1 -- 172.21.15.70:0/245549227 >> v1:172.21.15.70:6789/0 conn(0x7f71dc107690 legacy=0x7f71dc107a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.858+0000 7f71e0a21700 1 -- 172.21.15.70:0/245549227 shutdown_connections 2024-02-10T07:33:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.858+0000 7f71e0a21700 1 -- 172.21.15.70:0/245549227 >> 172.21.15.70:0/245549227 conn(0x7f71dc0fd130 msgr2=0x7f71dc0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.858+0000 7f71e0a21700 1 -- 172.21.15.70:0/245549227 shutdown_connections 2024-02-10T07:33:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.858+0000 7f71e0a21700 1 -- 172.21.15.70:0/245549227 wait complete. 2024-02-10T07:33:53.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.859+0000 7f71e0a21700 1 Processor -- start 2024-02-10T07:33:53.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.859+0000 7f71e0a21700 1 -- start start 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.859+0000 7f71e0a21700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f71dc075f10 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.859+0000 7f71da59c700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f71dc107690 0x7f71dc074020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55686/0 (socket says 172.21.15.70:55686) 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.859+0000 7f71da59c700 1 -- 172.21.15.70:0/485380989 learned_addr learned my addr 172.21.15.70:0/485380989 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.859+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497485326 0 0) 0x7f71dc075f10 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71c80054f0 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3794343409 0 0) 0x7f71c80054f0 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71dc075f10 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f71cc001980 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2387042753 0 0) 0x7f71dc075f10 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc074730 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc0749e0 con 0x7f71dc107690 2024-02-10T07:33:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc074e60 con 0x7f71dc107690 2024-02-10T07:33:53.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f71cc0011c0 con 0x7f71dc107690 2024-02-10T07:33:53.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.860+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f71cc004e60 con 0x7f71dc107690 2024-02-10T07:33:53.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.861+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7f71cc004fe0 con 0x7f71dc107690 2024-02-10T07:33:53.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.861+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc005120 con 0x7f71dc107690 2024-02-10T07:33:53.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.861+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1843170182 0 0) 0x7f71cc0472e0 con 0x7f71dc107690 2024-02-10T07:33:53.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:53.866+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f71cc016880 con 0x7f71dc107690 2024-02-10T07:33:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:53 smithi070 ceph-mon[29548]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:33:54.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.065+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f71bc0053d0 con 0x7f71dc107690 2024-02-10T07:33:54.805 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.805+0000 7f71d37fe700 1 -- 172.21.15.70:0/485380989 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (unknown 3126668360 0 0) 0x7f71cc014e80 con 0x7f71dc107690 2024-02-10T07:33:54.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.808+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 >> v1:172.21.15.70:6800/3858935040 conn(0x7f71c803a280 legacy=0x7f71c803c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:54.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.808+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 >> v1:172.21.15.70:6789/0 conn(0x7f71dc107690 legacy=0x7f71dc074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:54.808 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.808+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 shutdown_connections 2024-02-10T07:33:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.808+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 >> 172.21.15.70:0/485380989 conn(0x7f71dc0fd130 msgr2=0x7f71dc108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.808+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 shutdown_connections 2024-02-10T07:33:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:33:54.808+0000 7f71e0a21700 1 -- 172.21.15.70:0/485380989 wait complete. 2024-02-10T07:33:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/485380989' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:33:55.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:33:55.405 INFO:tasks.cephadm:Adding mon.a on smithi070 2024-02-10T07:33:55.405 INFO:tasks.cephadm:Adding mon.c on smithi070 2024-02-10T07:33:55.405 INFO:tasks.cephadm:Adding mon.b on smithi155 2024-02-10T07:33:55.405 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply mon '3;smithi070:[v1:172.21.15.70:6789]=a;smithi070:[v1:172.21.15.70:6790]=c;smithi155:[v1:172.21.15.155:6789]=b' 2024-02-10T07:33:55.642 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:33:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:55 smithi070 ceph-mon[29548]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:33:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:55 smithi070 ceph-mon[29548]: Updating smithi155:/etc/ceph/ceph.conf 2024-02-10T07:33:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/485380989' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T07:33:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:55 smithi070 ceph-mon[29548]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:33:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:55 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:55 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:55 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:57.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.194+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3728948467 >> v1:172.21.15.70:6789/0 conn(0x7f0c78102800 legacy=0x7f0c78102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:57.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.194+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3728948467 shutdown_connections 2024-02-10T07:33:57.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.194+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3728948467 >> 172.21.15.155:0/3728948467 conn(0x7f0c780fc090 msgr2=0x7f0c780fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.194+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3728948467 shutdown_connections 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.194+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3728948467 wait complete. 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.195+0000 7f0c7e2b4700 1 Processor -- start 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.195+0000 7f0c7e2b4700 1 -- start start 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.195+0000 7f0c7e2b4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c78198c00 con 0x7f0c78102800 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.195+0000 7f0c77fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0c78102800 0x7f0c781984f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.155:57834/0 (socket says 172.21.15.155:57834) 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.195+0000 7f0c77fff700 1 -- 172.21.15.155:0/3365048844 learned_addr learned my addr 172.21.15.155:0/3365048844 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.195+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339879457 0 0) 0x7f0c78198c00 con 0x7f0c78102800 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.195+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c600054f0 con 0x7f0c78102800 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4282663057 0 0) 0x7f0c600054f0 con 0x7f0c78102800 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c78198c00 con 0x7f0c78102800 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f0c680044b0 con 0x7f0c78102800 2024-02-10T07:33:57.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3649685230 0 0) 0x7f0c78198c00 con 0x7f0c78102800 2024-02-10T07:33:57.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c78198d90 con 0x7f0c78102800 2024-02-10T07:33:57.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c78199040 con 0x7f0c78102800 2024-02-10T07:33:57.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c781a1220 con 0x7f0c78102800 2024-02-10T07:33:57.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f0c68001620 con 0x7f0c78102800 2024-02-10T07:33:57.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.196+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7f0c68004e30 con 0x7f0c78102800 2024-02-10T07:33:57.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.197+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c64005120 con 0x7f0c78102800 2024-02-10T07:33:57.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.197+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7f0c68004fb0 con 0x7f0c78102800 2024-02-10T07:33:57.199 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.197+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2086688066 0 0) 0x7f0c680474a0 con 0x7f0c78102800 2024-02-10T07:33:57.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.202+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0c680168a0 con 0x7f0c78102800 2024-02-10T07:33:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:56 smithi070 ceph-mon[29548]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:33:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:56 smithi070 ceph-mon[29548]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:33:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:56 smithi070 ceph-mon[29548]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:33:57.413 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.413+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:[v1:172.21.15.70:6789]=a;smithi070:[v1:172.21.15.70:6790]=c;smithi155:[v1:172.21.15.155:6789]=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c64000c60 con 0x7f0c6003a290 2024-02-10T07:33:57.419 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.419+0000 7f0c757fa700 1 -- 172.21.15.155:0/3365048844 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f0c64000c60 con 0x7f0c6003a290 2024-02-10T07:33:57.419 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-02-10T07:33:57.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.422+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 >> v1:172.21.15.70:6800/3858935040 conn(0x7f0c6003a290 legacy=0x7f0c6003c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:57.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.422+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 >> v1:172.21.15.70:6789/0 conn(0x7f0c78102800 legacy=0x7f0c781984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:33:57.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.422+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 shutdown_connections 2024-02-10T07:33:57.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.422+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 >> 172.21.15.155:0/3365048844 conn(0x7f0c780fc090 msgr2=0x7f0c780fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:33:57.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.422+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 shutdown_connections 2024-02-10T07:33:57.422 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:33:57.422+0000 7f0c7e2b4700 1 -- 172.21.15.155:0/3365048844 wait complete. 2024-02-10T07:33:58.082 DEBUG:teuthology.orchestra.run.smithi070:mon.c> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.c.service 2024-02-10T07:33:58.085 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.b.service 2024-02-10T07:33:58.087 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T07:33:58.088 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T07:33:58.111 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:33:58.111 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:58.111 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:33:58.111 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:33:58.111 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:33:58.111 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:33:58.112 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:33:58.112 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:33:59.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:58 smithi070 ceph-mon[29548]: from='client.14184 v1:172.21.15.155:0/3365048844' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:[v1:172.21.15.70:6789]=a;smithi070:[v1:172.21.15.70:6790]=c;smithi155:[v1:172.21.15.155:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:33:59.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:58 smithi070 ceph-mon[29548]: Saving service mon spec with placement smithi070:[v1:172.21.15.70:6789]=a;smithi070:[v1:172.21.15.70:6790]=c;smithi155:[v1:172.21.15.155:6789]=b;count:3 2024-02-10T07:33:59.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:58 smithi070 ceph-mon[29548]: Deploying daemon mon.b on smithi155 2024-02-10T07:33:59.172 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:34:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:33:59 smithi070 ceph-mon[29548]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:01.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.902+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3128542393 >> v1:172.21.15.70:6789/0 conn(0x7fc6a81065e0 legacy=0x7fc6a81069c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:01.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.902+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3128542393 shutdown_connections 2024-02-10T07:34:01.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.902+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3128542393 >> 172.21.15.155:0/3128542393 conn(0x7fc6a80fc040 msgr2=0x7fc6a80fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:01.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.902+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3128542393 shutdown_connections 2024-02-10T07:34:01.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.902+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3128542393 wait complete. 2024-02-10T07:34:01.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.903+0000 7fc6ad1a3700 1 Processor -- start 2024-02-10T07:34:01.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.903+0000 7fc6ad1a3700 1 -- start start 2024-02-10T07:34:01.905 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.903+0000 7fc6ad1a3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6a81968d0 con 0x7fc6a81065e0 2024-02-10T07:34:01.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc6a6d9d700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc6a81065e0 0x7fc6a81961c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.155:57852/0 (socket says 172.21.15.155:57852) 2024-02-10T07:34:01.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc6a6d9d700 1 -- 172.21.15.155:0/3380886612 learned_addr learned my addr 172.21.15.155:0/3380886612 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:34:01.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233412918 0 0) 0x7fc6a81968d0 con 0x7fc6a81065e0 2024-02-10T07:34:01.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6940054f0 con 0x7fc6a81065e0 2024-02-10T07:34:01.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2969250105 0 0) 0x7fc6940054f0 con 0x7fc6a81065e0 2024-02-10T07:34:01.906 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6a81968d0 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc6a0001980 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1306082523 0 0) 0x7fc6a81968d0 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8196a60 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a8196d10 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a819ef70 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fc6a0001620 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.904+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 3289323883 0 0) 0x7fc6a0004e50 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.905+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7fc6a0004fd0 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.905+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc688005120 con 0x7fc6a81065e0 2024-02-10T07:34:01.907 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.906+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2086688066 0 0) 0x7fc6a0047490 con 0x7fc6a81065e0 2024-02-10T07:34:01.911 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:01.911+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc6a0005370 con 0x7fc6a81065e0 2024-02-10T07:34:02.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.200+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc6880053d0 con 0x7fc6a81065e0 2024-02-10T07:34:02.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.201+0000 7fc69ffff700 1 -- 172.21.15.155:0/3380886612 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+694 (unknown 2237029548 0 2618049802) 0x7fc6a0014e20 con 0x7fc6a81065e0 2024-02-10T07:34:02.202 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:34:02.202 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"70142e68-c7e6-11ee-95b8-87774f69a715","modified":"2024-02-10T07:32:24.003335Z","created":"2024-02-10T07:32:24.003335Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-10T07:34:02.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.203+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 >> v1:172.21.15.70:6800/3858935040 conn(0x7fc69403a290 legacy=0x7fc69403c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:02.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.203+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 >> v1:172.21.15.70:6789/0 conn(0x7fc6a81065e0 legacy=0x7fc6a81961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:02.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.203+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 shutdown_connections 2024-02-10T07:34:02.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.203+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 >> 172.21.15.155:0/3380886612 conn(0x7fc6a80fc040 msgr2=0x7fc6a8069950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:02.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.203+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 shutdown_connections 2024-02-10T07:34:02.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:02.203+0000 7fc6ad1a3700 1 -- 172.21.15.155:0/3380886612 wait complete. 2024-02-10T07:34:02.205 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-02-10T07:34:02.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:01 smithi070 ceph-mon[29548]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:04.002 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T07:34:04.003 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T07:34:04.246 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.551+0000 7fe65a6f8700 1 -- 172.21.15.155:0/325114913 >> v1:172.21.15.70:6789/0 conn(0x7fe654100730 legacy=0x7fe654100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.551+0000 7fe65a6f8700 1 -- 172.21.15.155:0/325114913 shutdown_connections 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.551+0000 7fe65a6f8700 1 -- 172.21.15.155:0/325114913 >> 172.21.15.155:0/325114913 conn(0x7fe6540fc190 msgr2=0x7fe6540fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.552+0000 7fe65a6f8700 1 -- 172.21.15.155:0/325114913 shutdown_connections 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.552+0000 7fe65a6f8700 1 -- 172.21.15.155:0/325114913 wait complete. 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.552+0000 7fe65a6f8700 1 Processor -- start 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.552+0000 7fe65a6f8700 1 -- start start 2024-02-10T07:34:07.554 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.553+0000 7fe65a6f8700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe6541a8de0 con 0x7fe6541a2f10 2024-02-10T07:34:07.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.553+0000 7fe65a6f8700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe6541a9f80 con 0x7fe654100730 2024-02-10T07:34:07.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.553+0000 7fe653fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe654100730 0x7fe6541a2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55718/0 (socket says 172.21.15.155:55718) 2024-02-10T07:34:07.555 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.553+0000 7fe653fff700 1 -- 172.21.15.155:0/2836738360 learned_addr learned my addr 172.21.15.155:0/2836738360 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:34:07.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.555+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796000884 0 0) 0x7fe6541a8de0 con 0x7fe6541a2f10 2024-02-10T07:34:07.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.555+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe63c0035b0 con 0x7fe6541a2f10 2024-02-10T07:34:07.556 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1682840661 0 0) 0x7fe63c0035b0 con 0x7fe6541a2f10 2024-02-10T07:34:07.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6541a8de0 con 0x7fe6541a2f10 2024-02-10T07:34:07.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 36600382 0 0) 0x7fe6480017e0 con 0x7fe6541a2f10 2024-02-10T07:34:07.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1450892106 0 0) 0x7fe6541a8de0 con 0x7fe6541a2f10 2024-02-10T07:34:07.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 >> v1:172.21.15.155:6789/0 conn(0x7fe654100730 legacy=0x7fe6541a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:07.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6541ab120 con 0x7fe6541a2f10 2024-02-10T07:34:07.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6541a8fd0 con 0x7fe6541a2f10 2024-02-10T07:34:07.558 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.556+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6541a9490 con 0x7fe6541a2f10 2024-02-10T07:34:07.558 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.557+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe654063350 con 0x7fe6541a2f10 2024-02-10T07:34:07.558 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.558+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fe648003b10 con 0x7fe6541a2f10 2024-02-10T07:34:07.559 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.558+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 36600382 0 0) 0x7fe6480048a0 con 0x7fe6541a2f10 2024-02-10T07:34:07.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.559+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7fe64800fdb0 con 0x7fe6541a2f10 2024-02-10T07:34:07.560 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.559+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2086688066 0 0) 0x7fe648045e10 con 0x7fe6541a2f10 2024-02-10T07:34:07.564 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.563+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fe6480116d0 con 0x7fe6541a2f10 2024-02-10T07:34:07.840 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.839+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe654112a50 con 0x7fe6541a2f10 2024-02-10T07:34:07.841 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:07.841 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.839+0000 7fe6517fa700 1 -- 172.21.15.155:0/2836738360 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+912 (unknown 3557084514 0 4041976839) 0x7fe6480137b0 con 0x7fe6541a2f10 2024-02-10T07:34:07.842 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:34:07.842 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"70142e68-c7e6-11ee-95b8-87774f69a715","modified":"2024-02-10T07:34:02.503423Z","created":"2024-02-10T07:32:24.003335Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-10T07:34:07.843 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.842+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 >> v1:172.21.15.70:6800/3858935040 conn(0x7fe63c03a3a0 legacy=0x7fe63c03c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:07.843 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.842+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 >> v1:172.21.15.70:6789/0 conn(0x7fe6541a2f10 legacy=0x7fe6541a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:07.843 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.842+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 shutdown_connections 2024-02-10T07:34:07.843 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.842+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 >> 172.21.15.155:0/2836738360 conn(0x7fe6540fc190 msgr2=0x7fe6540fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:07.843 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.842+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 shutdown_connections 2024-02-10T07:34:07.843 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:07.842+0000 7fe65a6f8700 1 -- 172.21.15.155:0/2836738360 wait complete. 2024-02-10T07:34:07.844 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-02-10T07:34:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:34:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: mon.a calling monitor election 2024-02-10T07:34:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: mon.b calling monitor election 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: monmap e2: 2 mons at {a=v1:172.21.15.70:6789/0,b=v1:172.21.15.155:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:34:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: fsmap 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: mgrmap e13: y(active, since 40s) 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: overall HEALTH_OK 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:07.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:08 smithi155 ceph-mon[32688]: Deploying daemon mon.c on smithi070 2024-02-10T07:34:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/2836738360' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:34:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:08 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:08 smithi070 ceph-mon[29548]: Deploying daemon mon.c on smithi070 2024-02-10T07:34:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/2836738360' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:34:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:08 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:09.393 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T07:34:09.394 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T07:34:09.642 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:34:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:09 smithi070 ceph-mon[29548]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:09 smithi155 ceph-mon[32688]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:11.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:11 smithi070 systemd[1]: Started Ceph mon.c for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.069+0000 7f9b4fb33700 1 -- 172.21.15.155:0/4228742102 >> v1:172.21.15.70:6789/0 conn(0x7f9b2c005800 legacy=0x7f9b2c007c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.069+0000 7f9b4fb33700 1 -- 172.21.15.155:0/4228742102 shutdown_connections 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.069+0000 7f9b4fb33700 1 -- 172.21.15.155:0/4228742102 >> 172.21.15.155:0/4228742102 conn(0x7f9b480fc180 msgr2=0x7f9b480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.070+0000 7f9b4fb33700 1 -- 172.21.15.155:0/4228742102 shutdown_connections 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.070+0000 7f9b4fb33700 1 -- 172.21.15.155:0/4228742102 wait complete. 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.070+0000 7f9b4fb33700 1 Processor -- start 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.070+0000 7f9b4fb33700 1 -- start start 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.071+0000 7f9b4fb33700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9b481a4150 con 0x7f9b48194900 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.071+0000 7f9b4fb33700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9b481a5330 con 0x7f9b2c005800 2024-02-10T07:34:16.072 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.071+0000 7f9b4fb33700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9b481a6530 con 0x7f9b481a0480 2024-02-10T07:34:16.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.071+0000 7f9b4d8cf700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f9b2c005800 0x7f9b481941f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55732/0 (socket says 172.21.15.155:55732) 2024-02-10T07:34:16.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.071+0000 7f9b4d8cf700 1 -- 172.21.15.155:0/901000279 learned_addr learned my addr 172.21.15.155:0/901000279 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:34:16.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.072+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666596768 0 0) 0x7f9b481a4150 con 0x7f9b48194900 2024-02-10T07:34:16.073 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.073+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b34003bb0 con 0x7f9b48194900 2024-02-10T07:34:16.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.073+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3061355955 0 0) 0x7f9b34003bb0 con 0x7f9b48194900 2024-02-10T07:34:16.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.073+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9b481a4150 con 0x7f9b48194900 2024-02-10T07:34:16.074 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.073+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f9b38001830 con 0x7f9b48194900 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3082931281 0 0) 0x7f9b481a4150 con 0x7f9b48194900 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 >> v1:172.21.15.70:6790/0 conn(0x7f9b481a0480 legacy=0x7f9b481a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 >> v1:172.21.15.155:6789/0 conn(0x7f9b2c005800 legacy=0x7f9b481941f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b481a7730 con 0x7f9b48194900 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b481a4340 con 0x7f9b48194900 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b481a4800 con 0x7f9b48194900 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f9b38003b10 con 0x7f9b48194900 2024-02-10T07:34:16.075 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.074+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f9b380049d0 con 0x7f9b48194900 2024-02-10T07:34:16.076 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.075+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7f9b3800fee0 con 0x7f9b48194900 2024-02-10T07:34:16.076 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.075+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2086688066 0 0) 0x7f9b38045f30 con 0x7f9b48194900 2024-02-10T07:34:16.076 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.075+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b48063350 con 0x7f9b48194900 2024-02-10T07:34:16.081 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.080+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f9b38011880 con 0x7f9b48194900 2024-02-10T07:34:16.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.367+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9b48195550 con 0x7f9b48194900 2024-02-10T07:34:16.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.367+0000 7f9b3effd700 1 -- 172.21.15.155:0/901000279 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1127 (unknown 409899127 0 2162273312) 0x7f9b38013960 con 0x7f9b48194900 2024-02-10T07:34:16.369 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-10T07:34:16.369 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":3,"fsid":"70142e68-c7e6-11ee-95b8-87774f69a715","modified":"2024-02-10T07:34:11.022393Z","created":"2024-02-10T07:32:24.003335Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6790","nonce":0}]},"addr":"172.21.15.70:6790/0","public_addr":"172.21.15.70:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: mon.a calling monitor election 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: mon.b calling monitor election 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: mon.c calling monitor election 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T07:34:16.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: monmap e3: 3 mons at {a=v1:172.21.15.70:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.70:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:34:16.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: fsmap 2024-02-10T07:34:16.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:34:16.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: mgrmap e13: y(active, since 49s) 2024-02-10T07:34:16.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: overall HEALTH_OK 2024-02-10T07:34:16.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:16.371 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:16.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.370+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 >> v1:172.21.15.70:6800/3858935040 conn(0x7f9b3403a180 legacy=0x7f9b3403c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:16.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.371+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 >> v1:172.21.15.70:6789/0 conn(0x7f9b48194900 legacy=0x7f9b4819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:16.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.371+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 shutdown_connections 2024-02-10T07:34:16.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.371+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 >> 172.21.15.155:0/901000279 conn(0x7f9b480fc180 msgr2=0x7f9b48104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:16.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.371+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 shutdown_connections 2024-02-10T07:34:16.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:16.371+0000 7f9b4fb33700 1 -- 172.21.15.155:0/901000279 wait complete. 2024-02-10T07:34:16.372 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 3 2024-02-10T07:34:16.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:34:16.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: mon.a calling monitor election 2024-02-10T07:34:16.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: mon.b calling monitor election 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: mon.c calling monitor election 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: monmap e3: 3 mons at {a=v1:172.21.15.70:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.70:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: fsmap 2024-02-10T07:34:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: mgrmap e13: y(active, since 49s) 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: overall HEALTH_OK 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: mon.a calling monitor election 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: mon.b calling monitor election 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:16.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: mon.c calling monitor election 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: monmap e3: 3 mons at {a=v1:172.21.15.70:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.70:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: fsmap 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: mgrmap e13: y(active, since 49s) 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: overall HEALTH_OK 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:16.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.045 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-10T07:34:17.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph config generate-minimal-conf 2024-02-10T07:34:17.326 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.326 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.327 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:17.327 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/901000279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:34:17.327 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:17 smithi155 ceph-mon[32688]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:17.327 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.327 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:17.424 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:34:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/901000279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:34:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[39923]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/901000279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:34:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[29548]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: Updating smithi070:/etc/ceph/ceph.conf 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: Updating smithi155:/etc/ceph/ceph.conf 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: Updating smithi070:/etc/ceph/ceph.conf 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: Updating smithi155:/etc/ceph/ceph.conf 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:18.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:18.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:18.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: Updating smithi070:/etc/ceph/ceph.conf 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: Updating smithi155:/etc/ceph/ceph.conf 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:18.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:18.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:18.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.074+0000 7f9a3c28b700 1 -- 172.21.15.70:0/3935817215 >> v1:172.21.15.70:6789/0 conn(0x7f9a341065e0 legacy=0x7f9a341069c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.074+0000 7f9a3c28b700 1 -- 172.21.15.70:0/3935817215 shutdown_connections 2024-02-10T07:34:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.074+0000 7f9a3c28b700 1 -- 172.21.15.70:0/3935817215 >> 172.21.15.70:0/3935817215 conn(0x7f9a340fc040 msgr2=0x7f9a340fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.074+0000 7f9a3c28b700 1 -- 172.21.15.70:0/3935817215 shutdown_connections 2024-02-10T07:34:19.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.074+0000 7f9a3c28b700 1 -- 172.21.15.70:0/3935817215 wait complete. 2024-02-10T07:34:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.075+0000 7f9a3c28b700 1 Processor -- start 2024-02-10T07:34:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.075+0000 7f9a3c28b700 1 -- start start 2024-02-10T07:34:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.075+0000 7f9a3c28b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a341a6320 con 0x7f9a341a2650 2024-02-10T07:34:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.075+0000 7f9a3c28b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a341a7500 con 0x7f9a34196ad0 2024-02-10T07:34:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.075+0000 7f9a3c28b700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a341a8700 con 0x7f9a341065e0 2024-02-10T07:34:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.075+0000 7f9a3a828700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9a341a2650 0x7f9a341a4a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56698/0 (socket says 172.21.15.70:56698) 2024-02-10T07:34:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.075+0000 7f9a3a828700 1 -- 172.21.15.70:0/1136394273 learned_addr learned my addr 172.21.15.70:0/1136394273 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920116018 0 0) 0x7f9a341a8700 con 0x7f9a341065e0 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a20003bb0 con 0x7f9a341065e0 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030490391 0 0) 0x7f9a341a6320 con 0x7f9a341a2650 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a341a8700 con 0x7f9a341a2650 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119051687 0 0) 0x7f9a341a7500 con 0x7f9a34196ad0 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a341a6320 con 0x7f9a34196ad0 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2565860957 0 0) 0x7f9a341a8700 con 0x7f9a341a2650 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a341a7500 con 0x7f9a341a2650 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f9a300017b0 con 0x7f9a341a2650 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 898106880 0 0) 0x7f9a20003bb0 con 0x7f9a341065e0 2024-02-10T07:34:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a341a8700 con 0x7f9a341065e0 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f9a2c001c80 con 0x7f9a341065e0 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4025660245 0 0) 0x7f9a341a6320 con 0x7f9a34196ad0 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a20003bb0 con 0x7f9a34196ad0 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f9a240017d0 con 0x7f9a34196ad0 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2308400778 0 0) 0x7f9a341a7500 con 0x7f9a341a2650 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 >> v1:172.21.15.70:6790/0 conn(0x7f9a341065e0 legacy=0x7f9a341963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 >> v1:172.21.15.155:6789/0 conn(0x7f9a34196ad0 legacy=0x7f9a341a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a341a9900 con 0x7f9a341a2650 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a341a6510 con 0x7f9a341a2650 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a341a6930 con 0x7f9a341a2650 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.076+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f9a30001d80 con 0x7f9a341a2650 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.077+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f9a30005b90 con 0x7f9a341a2650 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.077+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7f9a30005cf0 con 0x7f9a341a2650 2024-02-10T07:34:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.078+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a34063350 con 0x7f9a341a2650 2024-02-10T07:34:19.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.078+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2086688066 0 0) 0x7f9a300471b0 con 0x7f9a341a2650 2024-02-10T07:34:19.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.083+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f9a30007650 con 0x7f9a341a2650 2024-02-10T07:34:19.279 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.279+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9a34197720 con 0x7f9a341a2650 2024-02-10T07:34:19.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.279+0000 7f9a2b7fe700 1 -- 172.21.15.70:0/1136394273 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+191 (unknown 2189217731 0 1755566355) 0x7f9a300162c0 con 0x7f9a341a2650 2024-02-10T07:34:19.280 INFO:teuthology.orchestra.run.smithi070.stdout:# minimal ceph.conf for 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:34:19.280 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-02-10T07:34:19.280 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:34:19.280 INFO:teuthology.orchestra.run.smithi070.stdout: mon_host = 172.21.15.70:6789/0 172.21.15.155:6789/0 v1:172.21.15.70:6790/0 2024-02-10T07:34:19.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.281+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 >> v1:172.21.15.70:6800/3858935040 conn(0x7f9a2003a640 legacy=0x7f9a2003cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.281+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 >> v1:172.21.15.70:6789/0 conn(0x7f9a341a2650 legacy=0x7f9a341a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.281+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 shutdown_connections 2024-02-10T07:34:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.281+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 >> 172.21.15.70:0/1136394273 conn(0x7f9a340fc040 msgr2=0x7f9a34107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.281+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 shutdown_connections 2024-02-10T07:34:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:19.281+0000 7f9a3c28b700 1 -- 172.21.15.70:0/1136394273 wait complete. 2024-02-10T07:34:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[29548]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:34:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[29548]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:34:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[29548]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T07:34:19.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[29548]: Reconfiguring daemon mon.a on smithi070 2024-02-10T07:34:19.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[29548]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:19.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1136394273' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[39923]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:34:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[39923]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:34:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[39923]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T07:34:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[39923]: Reconfiguring daemon mon.a on smithi070 2024-02-10T07:34:19.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[39923]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:19.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1136394273' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:19 smithi155 ceph-mon[32688]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:34:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:19 smithi155 ceph-mon[32688]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:34:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:19 smithi155 ceph-mon[32688]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T07:34:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:19 smithi155 ceph-mon[32688]: Reconfiguring daemon mon.a on smithi070 2024-02-10T07:34:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:19 smithi155 ceph-mon[32688]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1136394273' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:20.168 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-10T07:34:20.169 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:34:20.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T07:34:20.199 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:34:20.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:34:20.267 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:34:20.267 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T07:34:20.301 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:34:20.302 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:34:20.372 INFO:tasks.cephadm:Adding mgr.y on smithi070 2024-02-10T07:34:20.372 INFO:tasks.cephadm:Adding mgr.x on smithi155 2024-02-10T07:34:20.372 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply mgr '2;smithi070=y;smithi155=x' 2024-02-10T07:34:20.648 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:34:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: Reconfiguring mon.c (monmap changed)... 2024-02-10T07:34:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: Reconfiguring daemon mon.c on smithi070 2024-02-10T07:34:21.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[39923]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:21.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:21.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:21.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: Reconfiguring mon.c (monmap changed)... 2024-02-10T07:34:21.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:21.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:21.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:21.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: Reconfiguring daemon mon.c on smithi070 2024-02-10T07:34:21.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:21 smithi070 ceph-mon[29548]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:21.841 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:21.842 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:21.842 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: Reconfiguring mon.c (monmap changed)... 2024-02-10T07:34:21.842 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:21.842 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:21.842 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:21.842 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: Reconfiguring daemon mon.c on smithi070 2024-02-10T07:34:21.842 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:21 smithi155 ceph-mon[32688]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:22.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.032+0000 7f805c48e700 1 -- 172.21.15.155:0/3781179448 >> v1:172.21.15.70:6789/0 conn(0x7f8054100700 legacy=0x7f8054100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:22.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.032+0000 7f805c48e700 1 -- 172.21.15.155:0/3781179448 shutdown_connections 2024-02-10T07:34:22.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.032+0000 7f805c48e700 1 -- 172.21.15.155:0/3781179448 >> 172.21.15.155:0/3781179448 conn(0x7f80540fbcc0 msgr2=0x7f80540fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:22.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.032+0000 7f805c48e700 1 -- 172.21.15.155:0/3781179448 shutdown_connections 2024-02-10T07:34:22.038 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.032+0000 7f805c48e700 1 -- 172.21.15.155:0/3781179448 wait complete. 2024-02-10T07:34:22.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.032+0000 7f805c48e700 1 Processor -- start 2024-02-10T07:34:22.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f805c48e700 1 -- start start 2024-02-10T07:34:22.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f805c48e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80541a1920 con 0x7f805419dc40 2024-02-10T07:34:22.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f805c48e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80541a2b20 con 0x7f805406a970 2024-02-10T07:34:22.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f805c48e700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80541a3d20 con 0x7f8054100700 2024-02-10T07:34:22.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f8059a29700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f805406a970 0x7f805419c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:38438/0 (socket says 172.21.15.155:38438) 2024-02-10T07:34:22.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f8059a29700 1 -- 172.21.15.155:0/595763747 learned_addr learned my addr 172.21.15.155:0/595763747 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f805aa2b700 1 --1- 172.21.15.155:0/595763747 >> v1:172.21.15.70:6789/0 conn(0x7f805419dc40 0x7f80541a0030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.155:49396/0 (socket says 172.21.15.155:49396) 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936792761 0 0) 0x7f80541a2b20 con 0x7f805406a970 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8040003bb0 con 0x7f805406a970 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907429655 0 0) 0x7f80541a1920 con 0x7f805419dc40 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80541a2b20 con 0x7f805419dc40 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328298334 0 0) 0x7f80541a3d20 con 0x7f8054100700 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80541a1920 con 0x7f8054100700 2024-02-10T07:34:22.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4038499694 0 0) 0x7f80541a2b20 con 0x7f805419dc40 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80541a3d20 con 0x7f805419dc40 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1376874390 0 0) 0x7f8040003bb0 con 0x7f805406a970 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80541a2b20 con 0x7f805406a970 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f804c001b30 con 0x7f805419dc40 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.033+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4285288182 0 0) 0x7f80541a1920 con 0x7f8054100700 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8040003bb0 con 0x7f8054100700 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f80440017b0 con 0x7f805406a970 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8050000fe0 con 0x7f8054100700 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2644645887 0 0) 0x7f80541a3d20 con 0x7f805419dc40 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 >> v1:172.21.15.70:6790/0 conn(0x7f8054100700 legacy=0x7f8054198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 >> v1:172.21.15.155:6789/0 conn(0x7f805406a970 legacy=0x7f805419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80541a4f20 con 0x7f805419dc40 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80541a1b10 con 0x7f805419dc40 2024-02-10T07:34:22.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f80541a2010 con 0x7f805419dc40 2024-02-10T07:34:22.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f804c002180 con 0x7f805419dc40 2024-02-10T07:34:22.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.034+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f804c005eb0 con 0x7f805419dc40 2024-02-10T07:34:22.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.035+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7f804c011480 con 0x7f805419dc40 2024-02-10T07:34:22.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.035+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2086688066 0 0) 0x7f804c0482e0 con 0x7f805419dc40 2024-02-10T07:34:22.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.035+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8038005150 con 0x7f805419dc40 2024-02-10T07:34:22.042 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.040+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f804c013010 con 0x7f805419dc40 2024-02-10T07:34:22.258 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.257+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi070=y;smithi155=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f8038000bc0 con 0x7f804003a570 2024-02-10T07:34:22.263 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.262+0000 7f804b7fe700 1 -- 172.21.15.155:0/595763747 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7f8038000bc0 con 0x7f804003a570 2024-02-10T07:34:22.263 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mgr update... 2024-02-10T07:34:22.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.264+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 >> v1:172.21.15.70:6800/3858935040 conn(0x7f804003a570 legacy=0x7f804003ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:22.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.264+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 >> v1:172.21.15.70:6789/0 conn(0x7f805419dc40 legacy=0x7f80541a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:22.265 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.264+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 shutdown_connections 2024-02-10T07:34:22.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.264+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 >> 172.21.15.155:0/595763747 conn(0x7f80540fbcc0 msgr2=0x7f80540fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:22.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.264+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 shutdown_connections 2024-02-10T07:34:22.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:34:22.264+0000 7f805c48e700 1 -- 172.21.15.155:0/595763747 wait complete. 2024-02-10T07:34:22.843 DEBUG:teuthology.orchestra.run.smithi155:mgr.x> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.x.service 2024-02-10T07:34:22.845 INFO:tasks.cephadm:Deploying OSDs... 2024-02-10T07:34:22.845 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:34:22.845 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:34:22.864 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T07:34:22.865 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-02-10T07:34:22.923 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T07:34:22.923 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:22.923 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-10T07:34:22.923 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:22.923 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:22.923 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-02-10 07:34:10.423315869 +0000 2024-02-10T07:34:22.923 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-02-10 07:28:23.844821987 +0000 2024-02-10T07:34:22.924 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-02-10 07:28:23.844821987 +0000 2024-02-10T07:34:22.924 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-02-10 07:28:23.844821987 +0000 2024-02-10T07:34:22.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T07:34:22.993 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-02-10T07:34:22.993 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-02-10T07:34:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000125306 s, 4.1 MB/s 2024-02-10T07:34:22.995 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T07:34:23.053 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-02-10T07:34:23.110 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T07:34:23.110 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:23.110 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-10T07:34:23.110 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:23.110 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:23.111 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-02-10 07:34:10.423315869 +0000 2024-02-10T07:34:23.111 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-02-10 07:28:24.183811891 +0000 2024-02-10T07:34:23.111 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-02-10 07:28:24.183811891 +0000 2024-02-10T07:34:23.111 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-02-10 07:28:24.183811891 +0000 2024-02-10T07:34:23.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: Reconfiguring mon.b (monmap changed)... 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: Reconfiguring daemon mon.b on smithi155 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: from='client.14208 v1:172.21.15.155:0/595763747' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi070=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: Saving service mgr spec with placement smithi070=y;smithi155=x;count:2 2024-02-10T07:34:23.144 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:23 smithi155 ceph-mon[32688]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:23.178 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-02-10T07:34:23.178 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-02-10T07:34:23.178 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000115025 s, 4.5 MB/s 2024-02-10T07:34:23.179 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T07:34:23.236 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: Reconfiguring mon.b (monmap changed)... 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: Reconfiguring daemon mon.b on smithi155 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: from='client.14208 v1:172.21.15.155:0/595763747' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi070=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: Saving service mgr spec with placement smithi070=y;smithi155=x;count:2 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[39923]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: Reconfiguring mon.b (monmap changed)... 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: Reconfiguring daemon mon.b on smithi155 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: from='client.14208 v1:172.21.15.155:0/595763747' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi070=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: Saving service mgr spec with placement smithi070=y;smithi155=x;count:2 2024-02-10T07:34:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:23.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:23 smithi070 ceph-mon[29548]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-02-10 07:34:10.423315869 +0000 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-02-10 07:28:24.531801527 +0000 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-02-10 07:28:24.531801527 +0000 2024-02-10T07:34:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-02-10 07:28:24.531801527 +0000 2024-02-10T07:34:23.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T07:34:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-02-10T07:34:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-02-10T07:34:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.8243e-05 s, 5.8 MB/s 2024-02-10T07:34:23.326 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T07:34:23.382 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-02-10T07:34:23.437 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T07:34:23.437 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:23.437 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-10T07:34:23.437 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:23.437 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:23.438 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-02-10 07:34:10.423315869 +0000 2024-02-10T07:34:23.438 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-02-10 07:28:24.855791878 +0000 2024-02-10T07:34:23.438 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-02-10 07:28:24.855791878 +0000 2024-02-10T07:34:23.438 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-02-10 07:28:24.855791878 +0000 2024-02-10T07:34:23.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T07:34:23.502 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-02-10T07:34:23.503 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-02-10T07:34:23.503 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000103567 s, 4.9 MB/s 2024-02-10T07:34:23.504 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T07:34:23.561 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:34:23.561 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:34:23.577 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T07:34:23.577 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-02-10T07:34:23.633 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-10 07:34:01.395950966 +0000 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-10 07:28:32.642891551 +0000 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-10 07:28:32.642891551 +0000 2024-02-10T07:34:23.634 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-02-10 07:28:32.642891551 +0000 2024-02-10T07:34:23.634 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T07:34:23.700 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-10T07:34:23.700 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-10T07:34:23.701 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000249802 s, 2.0 MB/s 2024-02-10T07:34:23.702 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T07:34:23.757 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-10 07:34:01.395950966 +0000 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-10 07:28:32.980884648 +0000 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-10 07:28:32.980884648 +0000 2024-02-10T07:34:23.813 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-02-10 07:28:32.980884648 +0000 2024-02-10T07:34:23.814 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T07:34:23.878 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-10T07:34:23.878 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-10T07:34:23.878 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000190182 s, 2.7 MB/s 2024-02-10T07:34:23.879 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T07:34:23.935 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-02-10T07:34:23.991 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T07:34:23.991 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:23.991 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-10T07:34:23.991 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:23.991 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:23.991 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-10 07:34:01.396950946 +0000 2024-02-10T07:34:23.991 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-10 07:28:33.305878009 +0000 2024-02-10T07:34:23.992 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-10 07:28:33.305878009 +0000 2024-02-10T07:34:23.992 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-02-10 07:28:33.305878009 +0000 2024-02-10T07:34:23.992 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T07:34:24.056 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-10T07:34:24.057 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-10T07:34:24.057 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000176418 s, 2.9 MB/s 2024-02-10T07:34:24.058 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T07:34:24.113 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-02-10T07:34:24.168 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-10 07:34:01.396950946 +0000 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-10 07:28:33.648871003 +0000 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-10 07:28:33.648871003 +0000 2024-02-10T07:34:24.169 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-02-10 07:28:33.648871003 +0000 2024-02-10T07:34:24.169 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T07:34:24.234 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-10T07:34:24.235 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-10T07:34:24.235 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000220149 s, 2.3 MB/s 2024-02-10T07:34:24.236 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T07:34:24.291 INFO:tasks.cephadm:Deploying osd.0 on smithi070 with /dev/vg_nvme/lv_4... 2024-02-10T07:34:24.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-10T07:34:24.535 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:34:25.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[29548]: Deploying daemon mgr.x on smithi155 2024-02-10T07:34:25.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:25.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:24 smithi070 ceph-mon[39923]: Deploying daemon mgr.x on smithi155 2024-02-10T07:34:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:25.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:34:25.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:34:25.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:34:25.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:25.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:24 smithi155 ceph-mon[32688]: Deploying daemon mgr.x on smithi155 2024-02-10T07:34:26.230 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:25 smithi155 ceph-mon[32688]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:26.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:25 smithi070 ceph-mon[29548]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:26.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:25 smithi070 ceph-mon[39923]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:26.243 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-10T07:34:26.243 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-10T07:34:26.244 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-02-10T07:34:26.244 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-02-10T07:34:26.244 INFO:teuthology.orchestra.run.smithi070.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00834214 s, 1.3 GB/s 2024-02-10T07:34:26.244 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 2024-02-10T07:34:26.244 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-02-10T07:34:26.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:26 smithi155 systemd[1]: Started Ceph mgr.x for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:34:26.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2024-02-10T07:34:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:34:27.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:26.876+0000 7f42aa395200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:34:27.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:26.973+0000 7f42aa395200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:34:27.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.651 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:27.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-mon[32688]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:27.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[29548]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[39923]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:27.997 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:27.650+0000 7f42aa395200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:34:27.997 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:27.743+0000 7f42aa395200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:34:27.997 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:27.900+0000 7f42aa395200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:34:27.998 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:27.997+0000 7f42aa395200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:34:28.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:28.081+0000 7f42aa395200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:34:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.638+0000 7fc0ad06b700 1 -- 172.21.15.70:0/2048871282 >> v1:172.21.15.70:6790/0 conn(0x7fc0a8108b80 legacy=0x7fc0a810b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.638+0000 7fc0ad06b700 1 -- 172.21.15.70:0/2048871282 shutdown_connections 2024-02-10T07:34:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.638+0000 7fc0ad06b700 1 -- 172.21.15.70:0/2048871282 >> 172.21.15.70:0/2048871282 conn(0x7fc0a80fbfb0 msgr2=0x7fc0a80fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.638+0000 7fc0ad06b700 1 -- 172.21.15.70:0/2048871282 shutdown_connections 2024-02-10T07:34:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.638+0000 7fc0ad06b700 1 -- 172.21.15.70:0/2048871282 wait complete. 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.639+0000 7fc0ad06b700 1 Processor -- start 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.639+0000 7fc0ad06b700 1 -- start start 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.639+0000 7fc0ad06b700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0a806a580 con 0x7fc0a8108b80 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.639+0000 7fc0ad06b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0a81a2fd0 con 0x7fc0a8100a10 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.639+0000 7fc0ad06b700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0a81a41b0 con 0x7fc0a8104e40 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc0a659c700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fc0a8104e40 0x7fc0a819d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:56594/0 (socket says 172.21.15.70:56594) 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc0a659c700 1 -- 172.21.15.70:0/1370371274 learned_addr learned my addr 172.21.15.70:0/1370371274 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742078376 0 0) 0x7fc0a806a580 con 0x7fc0a8108b80 2024-02-10T07:34:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0900049e0 con 0x7fc0a8108b80 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154982185 0 0) 0x7fc0a81a41b0 con 0x7fc0a8104e40 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a806a580 con 0x7fc0a8104e40 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491858191 0 0) 0x7fc0a81a2fd0 con 0x7fc0a8100a10 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a81a41b0 con 0x7fc0a8100a10 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2833179159 0 0) 0x7fc0a806a580 con 0x7fc0a8104e40 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0a81a2fd0 con 0x7fc0a8104e40 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc0980018f0 con 0x7fc0a8104e40 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2967560906 0 0) 0x7fc0900049e0 con 0x7fc0a8108b80 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0a806a580 con 0x7fc0a8108b80 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 512746421 0 0) 0x7fc0a81a41b0 con 0x7fc0a8100a10 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc0900049e0 con 0x7fc0a8100a10 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc0a0001ba0 con 0x7fc0a8108b80 2024-02-10T07:34:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc09c001050 con 0x7fc0a8100a10 2024-02-10T07:34:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1278232564 0 0) 0x7fc0a81a2fd0 con 0x7fc0a8104e40 2024-02-10T07:34:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 >> v1:172.21.15.155:6789/0 conn(0x7fc0a8100a10 legacy=0x7fc0a8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 >> v1:172.21.15.70:6789/0 conn(0x7fc0a8108b80 legacy=0x7fc0a81a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a81a5390 con 0x7fc0a8104e40 2024-02-10T07:34:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a81a43a0 con 0x7fc0a8104e40 2024-02-10T07:34:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a81a48a0 con 0x7fc0a8104e40 2024-02-10T07:34:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.640+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fc098001cc0 con 0x7fc0a8104e40 2024-02-10T07:34:28.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.641+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc098005d70 con 0x7fc0a8104e40 2024-02-10T07:34:28.644 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:28.349+0000 7f42aa395200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:34:28.644 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:28.460+0000 7f42aa395200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:34:28.644 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:28.546+0000 7f42aa395200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:34:28.645 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:28.640+0000 7f42aa395200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:34:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.641+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 13) v1 ==== 45953+0+0 (unknown 3079160328 0 0) 0x7fc098011260 con 0x7fc0a8104e40 2024-02-10T07:34:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.641+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc094005150 con 0x7fc0a8104e40 2024-02-10T07:34:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.641+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 2086688066 0 0) 0x7fc098047f80 con 0x7fc0a8104e40 2024-02-10T07:34:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.646+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc098012bc0 con 0x7fc0a8104e40 2024-02-10T07:34:28.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:28.854+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc094000bc0 con 0x7fc09003a400 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:34:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: Reconfiguring daemon mgr.y on smithi070 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T07:34:28.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:34:28.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:34:28.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:28.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: Reconfiguring daemon mgr.y on smithi070 2024-02-10T07:34:28.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:34:28.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:34:28.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:28.818+0000 7f42aa395200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:34:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:29.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: Reconfiguring daemon mgr.y on smithi070 2024-02-10T07:34:29.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:34:29.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:34:29.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:29.910 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[29548]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:29.910 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[29548]: from='client.34115 v1:172.21.15.70:0/1370371274' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[39923]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[39923]: from='client.34115 v1:172.21.15.70:0/1370371274' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:29.911 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:30.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:29.792+0000 7f42aa395200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:34:30.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:29.924+0000 7f42aa395200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:34:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:29 smithi155 ceph-mon[32688]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:29 smithi155 ceph-mon[32688]: from='client.34115 v1:172.21.15.70:0/1370371274' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:30.543 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:30.258+0000 7f42aa395200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:34:30.543 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:30.348+0000 7f42aa395200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:34:30.544 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:30.543+0000 7f42aa395200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:34:30.890 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:30.729+0000 7f42aa395200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:34:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:30 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:30 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:30 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:30 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:30 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:30 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:31.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:31.012+0000 7f42aa395200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:34:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:30 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:30 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:30 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:31.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:31.271+0000 7f42aa395200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:34:31.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:31.505+0000 7f42aa395200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:34:32.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:31 smithi070 ceph-mon[39923]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:32.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:31 smithi070 ceph-mon[29548]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:31 smithi155 ceph-mon[32688]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:32.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:32.308+0000 7f42aa395200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:34:33.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2923516249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21f7c994-f25f-4bfe-ba6b-bcb06c637113"}]: dispatch 2024-02-10T07:34:33.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2923516249' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21f7c994-f25f-4bfe-ba6b-bcb06c637113"}]': finished 2024-02-10T07:34:33.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-mon[32688]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T07:34:33.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:33.195 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1732288764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:34:33.195 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:32.881+0000 7f42aa395200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:34:33.196 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:32.981+0000 7f42aa395200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:34:33.196 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:34:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:34:33.195+0000 7f42aa395200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:34:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2923516249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21f7c994-f25f-4bfe-ba6b-bcb06c637113"}]: dispatch 2024-02-10T07:34:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2923516249' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21f7c994-f25f-4bfe-ba6b-bcb06c637113"}]': finished 2024-02-10T07:34:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[29548]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T07:34:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1732288764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:34:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2923516249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21f7c994-f25f-4bfe-ba6b-bcb06c637113"}]: dispatch 2024-02-10T07:34:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2923516249' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21f7c994-f25f-4bfe-ba6b-bcb06c637113"}]': finished 2024-02-10T07:34:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[39923]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T07:34:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1732288764' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:34:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:33.974+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mgrmap(e 14) v1 ==== 91315+0+0 (unknown 25461725 0 0) 0x7fc098046880 con 0x7fc0a8104e40 2024-02-10T07:34:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[29548]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[29548]: Standby manager daemon x started 2024-02-10T07:34:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:34:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:34:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:34:34.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:34:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[39923]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[39923]: Standby manager daemon x started 2024-02-10T07:34:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:34:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:34:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:34:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:33 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:34:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:33 smithi155 ceph-mon[32688]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:33 smithi155 ceph-mon[32688]: Standby manager daemon x started 2024-02-10T07:34:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:33 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:34:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:33 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:34:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:33 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:34:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:33 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/430288409' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:34:35.140 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:34 smithi070 ceph-mon[29548]: mgrmap e14: y(active, since 67s), standbys: x 2024-02-10T07:34:35.141 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:34 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:34:35.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:34 smithi070 ceph-mon[39923]: mgrmap e14: y(active, since 67s), standbys: x 2024-02-10T07:34:35.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:34 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:34:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:34 smithi155 ceph-mon[32688]: mgrmap e14: y(active, since 67s), standbys: x 2024-02-10T07:34:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:34 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:34:36.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:35 smithi070 ceph-mon[29548]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:36.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:35 smithi070 ceph-mon[39923]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:36.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:35 smithi155 ceph-mon[32688]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:37 smithi155 ceph-mon[32688]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:37.913 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:37 smithi070 ceph-mon[29548]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:37.914 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:37 smithi070 ceph-mon[39923]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:38 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T07:34:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:38 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T07:34:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:38 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T07:34:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:38 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:39.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:39 smithi070 ceph-mon[39923]: Deploying daemon osd.0 on smithi070 2024-02-10T07:34:39.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:39 smithi070 ceph-mon[39923]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:39 smithi070 ceph-mon[29548]: Deploying daemon osd.0 on smithi070 2024-02-10T07:34:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:39 smithi070 ceph-mon[29548]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:39 smithi155 ceph-mon[32688]: Deploying daemon osd.0 on smithi070 2024-02-10T07:34:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:39 smithi155 ceph-mon[32688]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:42.148 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:41 smithi070 ceph-mon[29548]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:42.149 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:41 smithi070 ceph-mon[39923]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:41 smithi155 ceph-mon[32688]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:43 smithi070 ceph-mon[29548]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:43 smithi070 ceph-mon[39923]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:43 smithi155 ceph-mon[32688]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:44 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:44 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:44 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:45.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:44 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:45.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:44 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:44 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:44 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:44 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:44 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:46.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:45 smithi070 ceph-mon[29548]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:46.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:45 smithi070 ceph-mon[39923]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:45 smithi155 ceph-mon[32688]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:46.958 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:46.957+0000 7fc08ffff700 1 -- 172.21.15.70:0/1370371274 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 93413307) 0x7fc094000bc0 con 0x7fc09003a400 2024-02-10T07:34:46.958 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 0 on host 'smithi070' 2024-02-10T07:34:46.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:46.959+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 >> v1:172.21.15.70:6800/3858935040 conn(0x7fc09003a400 legacy=0x7fc09003c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:46.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:46.959+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 >> v1:172.21.15.70:6790/0 conn(0x7fc0a8104e40 legacy=0x7fc0a819d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:46.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:46.959+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 shutdown_connections 2024-02-10T07:34:46.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:46.959+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 >> 172.21.15.70:0/1370371274 conn(0x7fc0a80fbfb0 msgr2=0x7fc0a81079f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:46.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:46.959+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 shutdown_connections 2024-02-10T07:34:46.960 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:46.959+0000 7fc0ad06b700 1 -- 172.21.15.70:0/1370371274 wait complete. 2024-02-10T07:34:47.956 DEBUG:teuthology.orchestra.run.smithi070:osd.0> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.0.service 2024-02-10T07:34:47.959 INFO:tasks.cephadm:Deploying osd.1 on smithi070 with /dev/vg_nvme/lv_3... 2024-02-10T07:34:47.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-10T07:34:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[29548]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[29548]: from='osd.0 v1:172.21.15.70:6801/3723357830' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:34:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[29548]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:34:48.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[39923]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[39923]: from='osd.0 v1:172.21.15.70:6801/3723357830' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:34:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:47 smithi070 ceph-mon[39923]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:34:48.248 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:34:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:47 smithi155 ceph-mon[32688]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:47 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:47 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:47 smithi155 ceph-mon[32688]: from='osd.0 v1:172.21.15.70:6801/3723357830' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:34:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:47 smithi155 ceph-mon[32688]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:34:49.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[39923]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T07:34:49.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[39923]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[39923]: from='osd.0 v1:172.21.15.70:6801/3723357830' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[39923]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[29548]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[29548]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[29548]: from='osd.0 v1:172.21.15.70:6801/3723357830' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[29548]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:34:49.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:49.187 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:48 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:49.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:48 smithi155 ceph-mon[32688]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T07:34:49.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:48 smithi155 ceph-mon[32688]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T07:34:49.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:48 smithi155 ceph-mon[32688]: from='osd.0 v1:172.21.15.70:6801/3723357830' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:34:49.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:48 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:49.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:48 smithi155 ceph-mon[32688]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:34:49.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:48 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:49.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:48 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:49.468 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0[43929]: 2024-02-10T07:34:49.184+0000 7f59aea23700 -1 osd.0 0 waiting for initial osdmap 2024-02-10T07:34:49.477 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0[43929]: 2024-02-10T07:34:49.186+0000 7f59a680e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:34:50.057 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[39923]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[39923]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[39923]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[39923]: from='osd.0 ' entity='osd.0' 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[29548]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[29548]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[29548]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:49 smithi070 ceph-mon[29548]: from='osd.0 ' entity='osd.0' 2024-02-10T07:34:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-10T07:34:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-10T07:34:50.170 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-02-10T07:34:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-02-10T07:34:50.170 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00975157 s, 1.1 GB/s 2024-02-10T07:34:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-02-10T07:34:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:49 smithi155 ceph-mon[32688]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:49 smithi155 ceph-mon[32688]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:34:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:49 smithi155 ceph-mon[32688]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T07:34:50.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:49 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:49 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:49 smithi155 ceph-mon[32688]: from='osd.0 ' entity='osd.0' 2024-02-10T07:34:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:34:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:34:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[39923]: osd.0 v1:172.21.15.70:6801/3723357830 boot 2024-02-10T07:34:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[39923]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T07:34:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:34:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:34:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[29548]: osd.0 v1:172.21.15.70:6801/3723357830 boot 2024-02-10T07:34:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[29548]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T07:34:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:50 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:51.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2024-02-10T07:34:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:50 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:34:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:50 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:34:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:50 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:50 smithi155 ceph-mon[32688]: osd.0 v1:172.21.15.70:6801/3723357830 boot 2024-02-10T07:34:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:50 smithi155 ceph-mon[32688]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T07:34:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:50 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:34:52.087 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:34:52.212 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:51 smithi070 ceph-mon[29548]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:52.213 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:51 smithi070 ceph-mon[39923]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:51 smithi155 ceph-mon[32688]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:34:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:52 smithi070 ceph-mon[29548]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T07:34:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:52 smithi070 ceph-mon[39923]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T07:34:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:52 smithi155 ceph-mon[32688]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T07:34:54.161 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:54.161 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.161 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.161 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:34:54.161 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.161 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:54.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:54.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:54.162 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:54.162 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:34:54.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:54.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:54.164 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.164 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:34:54.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.378+0000 7f4b699c6700 1 -- 172.21.15.70:0/2701511977 >> v1:172.21.15.70:6790/0 conn(0x7f4b64108ba0 legacy=0x7f4b6410b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:54.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.378+0000 7f4b699c6700 1 -- 172.21.15.70:0/2701511977 shutdown_connections 2024-02-10T07:34:54.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.378+0000 7f4b699c6700 1 -- 172.21.15.70:0/2701511977 >> 172.21.15.70:0/2701511977 conn(0x7f4b640fc010 msgr2=0x7f4b640fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:34:54.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.379+0000 7f4b699c6700 1 -- 172.21.15.70:0/2701511977 shutdown_connections 2024-02-10T07:34:54.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.379+0000 7f4b699c6700 1 -- 172.21.15.70:0/2701511977 wait complete. 2024-02-10T07:34:54.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.379+0000 7f4b699c6700 1 Processor -- start 2024-02-10T07:34:54.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.379+0000 7f4b699c6700 1 -- start start 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b699c6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b64197900 con 0x7f4b64108ba0 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b699c6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b641a30b0 con 0x7f4b64104e60 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b699c6700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b641a4250 con 0x7f4b64100a30 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b62ffd700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f4b64100a30 0x7f4b64196db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:36840/0 (socket says 172.21.15.70:36840) 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b62ffd700 1 -- 172.21.15.70:0/1085743990 learned_addr learned my addr 172.21.15.70:0/1085743990 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471127009 0 0) 0x7f4b64197900 con 0x7f4b64108ba0 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b4c0035b0 con 0x7f4b64108ba0 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123682752 0 0) 0x7f4b641a4250 con 0x7f4b64100a30 2024-02-10T07:34:54.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b64197900 con 0x7f4b64100a30 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930978065 0 0) 0x7f4b641a30b0 con 0x7f4b64104e60 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b641a4250 con 0x7f4b64104e60 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4257704321 0 0) 0x7f4b4c0035b0 con 0x7f4b64108ba0 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b641a30b0 con 0x7f4b64108ba0 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4b5c0017b0 con 0x7f4b64108ba0 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1361645740 0 0) 0x7f4b64197900 con 0x7f4b64100a30 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b4c0035b0 con 0x7f4b64100a30 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4b54000fe0 con 0x7f4b64100a30 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3630472688 0 0) 0x7f4b641a4250 con 0x7f4b64104e60 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b64197900 con 0x7f4b64104e60 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4b58001830 con 0x7f4b64104e60 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4000533882 0 0) 0x7f4b641a30b0 con 0x7f4b64108ba0 2024-02-10T07:34:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 >> v1:172.21.15.70:6790/0 conn(0x7f4b64100a30 legacy=0x7f4b64196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 >> v1:172.21.15.155:6789/0 conn(0x7f4b64104e60 legacy=0x7f4b6419d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b641a53f0 con 0x7f4b64108ba0 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.380+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b641a4440 con 0x7f4b64108ba0 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.381+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b641a4940 con 0x7f4b64108ba0 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.381+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f4b5c004910 con 0x7f4b64108ba0 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.381+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4b5c0050d0 con 0x7f4b64108ba0 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.382+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b50005150 con 0x7f4b64108ba0 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.382+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 14) v1 ==== 91315+0+0 (unknown 25461725 0 0) 0x7f4b5c003a00 con 0x7f4b64108ba0 2024-02-10T07:34:54.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.382+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(9..9 src has 1..9) v4 ==== 1541+0+0 (unknown 15535047 0 0) 0x7f4b5c086980 con 0x7f4b64108ba0 2024-02-10T07:34:54.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.387+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f4b5c0513b0 con 0x7f4b64108ba0 2024-02-10T07:34:54.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:34:54.590+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b50000bc0 con 0x7f4b4c06eb00 2024-02-10T07:34:55.202 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi070 2024-02-10T07:34:55.202 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi070 to 16064M 2024-02-10T07:34:55.202 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi070 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi070 to 16064M 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:55.203 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:34:55.204 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:34:55.204 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi070 2024-02-10T07:34:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi070 to 16064M 2024-02-10T07:34:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:34:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:34:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:34:55.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:34:55.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:34:56.208 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:55 smithi070 ceph-mon[29548]: from='client.14247 v1:172.21.15.70:0/1085743990' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:56.208 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:55 smithi070 ceph-mon[29548]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:56.208 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:55 smithi070 ceph-mon[39923]: from='client.14247 v1:172.21.15.70:0/1085743990' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:56.208 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:55 smithi070 ceph-mon[39923]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:55 smithi155 ceph-mon[32688]: from='client.14247 v1:172.21.15.70:0/1085743990' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:34:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:55 smithi155 ceph-mon[32688]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[39923]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132693771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4164b54-8dde-4071-846d-959548c7fbe9"}]: dispatch 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132693771' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4164b54-8dde-4071-846d-959548c7fbe9"}]': finished 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[39923]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[29548]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132693771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4164b54-8dde-4071-846d-959548c7fbe9"}]: dispatch 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132693771' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4164b54-8dde-4071-846d-959548c7fbe9"}]': finished 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[29548]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T07:34:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:34:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:57 smithi155 ceph-mon[32688]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132693771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4164b54-8dde-4071-846d-959548c7fbe9"}]: dispatch 2024-02-10T07:34:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132693771' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4164b54-8dde-4071-846d-959548c7fbe9"}]': finished 2024-02-10T07:34:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:57 smithi155 ceph-mon[32688]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T07:34:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:34:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1564850211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:34:58.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1564850211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:34:59.004 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1564850211' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:34:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:34:59 smithi070 ceph-mon[29548]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:34:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:34:59 smithi070 ceph-mon[39923]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:34:59 smithi155 ceph-mon[32688]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:01 smithi070 ceph-mon[39923]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:01 smithi070 ceph-mon[29548]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:01 smithi155 ceph-mon[32688]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:04.219 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:03 smithi070 ceph-mon[29548]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:04.219 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T07:35:04.219 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:04.219 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:03 smithi070 ceph-mon[39923]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:04.219 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T07:35:04.219 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:03 smithi155 ceph-mon[32688]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T07:35:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:05.159 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:04 smithi070 ceph-mon[29548]: Deploying daemon osd.1 on smithi070 2024-02-10T07:35:05.160 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:04 smithi070 ceph-mon[39923]: Deploying daemon osd.1 on smithi070 2024-02-10T07:35:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:04 smithi155 ceph-mon[32688]: Deploying daemon osd.1 on smithi070 2024-02-10T07:35:06.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:05 smithi155 ceph-mon[32688]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:06.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:05 smithi070 ceph-mon[39923]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:06.174 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:05 smithi070 ceph-mon[29548]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:07 smithi070 ceph-mon[29548]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:07 smithi070 ceph-mon[39923]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:07 smithi155 ceph-mon[32688]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[29548]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:10.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[39923]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:09 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:10.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:09 smithi155 ceph-mon[32688]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:10.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:09 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:10.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:09 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:10.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:09 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:12.206 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:11 smithi070 ceph-mon[29548]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:12.206 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:11 smithi070 ceph-mon[39923]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:11 smithi155 ceph-mon[32688]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:12.341+0000 7f4b689c4700 1 -- 172.21.15.70:0/1085743990 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1501868303) 0x7f4b50000bc0 con 0x7f4b4c06eb00 2024-02-10T07:35:12.342 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 1 on host 'smithi070' 2024-02-10T07:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:12.344+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 >> v1:172.21.15.70:6800/3858935040 conn(0x7f4b4c06eb00 legacy=0x7f4b4c070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:12.344+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 >> v1:172.21.15.70:6789/0 conn(0x7f4b64108ba0 legacy=0x7f4b641a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:12.344+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 shutdown_connections 2024-02-10T07:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:12.344+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 >> 172.21.15.70:0/1085743990 conn(0x7f4b640fc010 msgr2=0x7f4b6406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:35:12.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:12.345+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 shutdown_connections 2024-02-10T07:35:12.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:12.345+0000 7f4b699c6700 1 -- 172.21.15.70:0/1085743990 wait complete. 2024-02-10T07:35:13.019 DEBUG:teuthology.orchestra.run.smithi070:osd.1> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.1.service 2024-02-10T07:35:13.021 INFO:tasks.cephadm:Deploying osd.2 on smithi070 with /dev/vg_nvme/lv_2... 2024-02-10T07:35:13.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-10T07:35:13.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[39923]: from='osd.1 v1:172.21.15.70:6805/918939980' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:35:13.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[39923]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:35:13.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.178 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[29548]: from='osd.1 v1:172.21.15.70:6805/918939980' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:35:13.178 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[29548]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:35:13.178 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.178 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.179 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.179 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:12 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:12 smithi155 ceph-mon[32688]: from='osd.1 v1:172.21.15.70:6805/918939980' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:35:13.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:12 smithi155 ceph-mon[32688]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:35:13.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:12 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:12 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:12 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:13.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:12 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:14.047 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[29548]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[29548]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[29548]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[29548]: from='osd.1 v1:172.21.15.70:6805/918939980' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[29548]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[39923]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:14.162 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[39923]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T07:35:14.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[39923]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T07:35:14.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:14.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[39923]: from='osd.1 v1:172.21.15.70:6805/918939980' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:14.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-mon[39923]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:14.163 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1[49779]: 2024-02-10T07:35:13.876+0000 7fa89ad3e700 -1 osd.1 0 waiting for initial osdmap 2024-02-10T07:35:14.163 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:13 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1[49779]: 2024-02-10T07:35:13.878+0000 7fa896b31700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:35:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:13 smithi155 ceph-mon[32688]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:13 smithi155 ceph-mon[32688]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T07:35:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:13 smithi155 ceph-mon[32688]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T07:35:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:13 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:13 smithi155 ceph-mon[32688]: from='osd.1 v1:172.21.15.70:6805/918939980' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:13 smithi155 ceph-mon[32688]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:15.001 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[29548]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:35:15.001 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[29548]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T07:35:15.001 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:15.002 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:15.002 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[39923]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:35:15.002 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[39923]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T07:35:15.002 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:15.002 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:14 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:15.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:14 smithi155 ceph-mon[32688]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:35:15.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:14 smithi155 ceph-mon[32688]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T07:35:15.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:14 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:15.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:14 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:16.142 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:35:16.142 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[39923]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[39923]: osd.1 v1:172.21.15.70:6805/918939980 boot 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[39923]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[29548]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[29548]: osd.1 v1:172.21.15.70:6805/918939980 boot 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[29548]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T07:35:16.143 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:15 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:15 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:35:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:15 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:35:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:15 smithi155 ceph-mon[32688]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:35:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:15 smithi155 ceph-mon[32688]: osd.1 v1:172.21.15.70:6805/918939980 boot 2024-02-10T07:35:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:15 smithi155 ceph-mon[32688]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T07:35:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:15 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:35:16.575 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-10T07:35:16.575 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-10T07:35:16.575 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-02-10T07:35:16.575 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-02-10T07:35:16.575 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00968367 s, 1.1 GB/s 2024-02-10T07:35:16.575 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-02-10T07:35:17.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:17.191 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:17.192 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:17.193 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.218 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2024-02-10T07:35:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T07:35:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:17.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:17.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:17.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:17.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:17.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:17.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:17.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:35:18.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:17 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi070 2024-02-10T07:35:18.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:17 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi070 to 8032M 2024-02-10T07:35:18.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:17 smithi070 ceph-mon[29548]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:18.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:17 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi070 2024-02-10T07:35:18.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:17 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi070 to 8032M 2024-02-10T07:35:18.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:17 smithi070 ceph-mon[39923]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:17 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi070 2024-02-10T07:35:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:17 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi070 to 8032M 2024-02-10T07:35:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:17 smithi155 ceph-mon[32688]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:19.019 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.019+0000 7f86d33c9700 1 -- 172.21.15.70:0/3659715361 >> v1:172.21.15.70:6789/0 conn(0x7f86cc109c80 legacy=0x7f86cc10c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.019+0000 7f86d33c9700 1 -- 172.21.15.70:0/3659715361 shutdown_connections 2024-02-10T07:35:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.019+0000 7f86d33c9700 1 -- 172.21.15.70:0/3659715361 >> 172.21.15.70:0/3659715361 conn(0x7f86cc0fd090 msgr2=0x7f86cc0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:35:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.019+0000 7f86d33c9700 1 -- 172.21.15.70:0/3659715361 shutdown_connections 2024-02-10T07:35:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.019+0000 7f86d33c9700 1 -- 172.21.15.70:0/3659715361 wait complete. 2024-02-10T07:35:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.019+0000 7f86d33c9700 1 Processor -- start 2024-02-10T07:35:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86d33c9700 1 -- start start 2024-02-10T07:35:19.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86d33c9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f86cc1a3f90 con 0x7f86cc105f40 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86d33c9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f86cc1a5190 con 0x7f86cc101b10 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86d33c9700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f86cc1a6390 con 0x7f86cc109c80 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86d0964700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86cc105f40 0x7f86cc19ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45062/0 (socket says 172.21.15.70:45062) 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86d0964700 1 -- 172.21.15.70:0/3002710775 learned_addr learned my addr 172.21.15.70:0/3002710775 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440128259 0 0) 0x7f86cc1a3f90 con 0x7f86cc105f40 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b80037f0 con 0x7f86cc105f40 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096220491 0 0) 0x7f86cc1a6390 con 0x7f86cc109c80 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86cc1a3f90 con 0x7f86cc109c80 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2874953377 0 0) 0x7f86b80037f0 con 0x7f86cc105f40 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86cc1a6390 con 0x7f86cc105f40 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f86bc004090 con 0x7f86cc105f40 2024-02-10T07:35:19.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4149015981 0 0) 0x7f86cc1a3f90 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86b80037f0 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f86c4004810 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978447643 0 0) 0x7f86cc1a5190 con 0x7f86cc101b10 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.020+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86cc1a3f90 con 0x7f86cc101b10 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3600374540 0 0) 0x7f86b80037f0 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 >> v1:172.21.15.155:6789/0 conn(0x7f86cc101b10 legacy=0x7f86cc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 >> v1:172.21.15.70:6789/0 conn(0x7f86cc105f40 legacy=0x7f86cc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc1a7590 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc1a6580 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc1a6a80 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f86c4001ef0 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.021+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f86c4005130 con 0x7f86cc109c80 2024-02-10T07:35:19.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.022+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b0005150 con 0x7f86cc109c80 2024-02-10T07:35:19.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.022+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 14) v1 ==== 91315+0+0 (unknown 25461725 0 0) 0x7f86c4003be0 con 0x7f86cc109c80 2024-02-10T07:35:19.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.022+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 1833+0+0 (unknown 3990418848 0 0) 0x7f86c4086a20 con 0x7f86cc109c80 2024-02-10T07:35:19.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.027+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f86c4051390 con 0x7f86cc109c80 2024-02-10T07:35:19.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:19.228+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f86b0000bc0 con 0x7f86b806e9e0 2024-02-10T07:35:20.135 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[29548]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:20.135 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:35:20.135 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:35:20.135 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:20.136 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[39923]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:20.136 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:35:20.136 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:35:20.136 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:19 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:19 smithi155 ceph-mon[32688]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:19 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:35:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:19 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:35:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:19 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:20 smithi070 ceph-mon[39923]: from='client.34169 v1:172.21.15.70:0/3002710775' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:35:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:20 smithi070 ceph-mon[29548]: from='client.34169 v1:172.21.15.70:0/3002710775' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:35:21.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:20 smithi155 ceph-mon[32688]: from='client.34169 v1:172.21.15.70:0/3002710775' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:35:22.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[29548]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:22.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3751747171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]: dispatch 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[29548]: from='client.34175 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]: dispatch 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[29548]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]': finished 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[29548]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[39923]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3751747171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]: dispatch 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[39923]: from='client.34175 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]: dispatch 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[39923]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]': finished 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[39923]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T07:35:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:21 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:21 smithi155 ceph-mon[32688]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:22.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3751747171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]: dispatch 2024-02-10T07:35:22.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:21 smithi155 ceph-mon[32688]: from='client.34175 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]: dispatch 2024-02-10T07:35:22.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:21 smithi155 ceph-mon[32688]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5ea7978-5f0f-4ba4-a881-ed48efdd57d1"}]': finished 2024-02-10T07:35:22.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:21 smithi155 ceph-mon[32688]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T07:35:22.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:21 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:23.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2705009899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:35:23.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2705009899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:35:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2705009899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:35:24.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:23 smithi070 ceph-mon[29548]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:24.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:23 smithi070 ceph-mon[39923]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:24.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:23 smithi155 ceph-mon[32688]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:26.150 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:25 smithi070 ceph-mon[29548]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:26.151 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:25 smithi070 ceph-mon[39923]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:25 smithi155 ceph-mon[32688]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:27 smithi070 ceph-mon[29548]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:27 smithi070 ceph-mon[39923]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:27 smithi155 ceph-mon[32688]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:28.984 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T07:35:28.984 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:28.984 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T07:35:28.984 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T07:35:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:29 smithi070 ceph-mon[29548]: Deploying daemon osd.2 on smithi070 2024-02-10T07:35:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:29 smithi070 ceph-mon[29548]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:29 smithi070 ceph-mon[39923]: Deploying daemon osd.2 on smithi070 2024-02-10T07:35:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:29 smithi070 ceph-mon[39923]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:29 smithi155 ceph-mon[32688]: Deploying daemon osd.2 on smithi070 2024-02-10T07:35:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:29 smithi155 ceph-mon[32688]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:31 smithi070 ceph-mon[29548]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:31 smithi070 ceph-mon[39923]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:31 smithi155 ceph-mon[32688]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:34.221 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[39923]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:34.222 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:34.222 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:34.222 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:34.222 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[29548]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:34.222 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:34.222 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:34.222 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:33 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:33 smithi155 ceph-mon[32688]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:33 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:33 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:33 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:35 smithi070 ceph-mon[29548]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:35 smithi070 ceph-mon[39923]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:36.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:35 smithi155 ceph-mon[32688]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:37.684 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 2 on host 'smithi070' 2024-02-10T07:35:37.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:37.683+0000 7f86c27fc700 1 -- 172.21.15.70:0/3002710775 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3183117011) 0x7f86b0000bc0 con 0x7f86b806e9e0 2024-02-10T07:35:37.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:37.685+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 >> v1:172.21.15.70:6800/3858935040 conn(0x7f86b806e9e0 legacy=0x7f86b8070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:37.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:37.685+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 >> v1:172.21.15.70:6790/0 conn(0x7f86cc109c80 legacy=0x7f86cc1a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:37.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:37.685+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 shutdown_connections 2024-02-10T07:35:37.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:37.685+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 >> 172.21.15.70:0/3002710775 conn(0x7f86cc0fd090 msgr2=0x7f86cc10c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:35:37.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:37.685+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 shutdown_connections 2024-02-10T07:35:37.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:37.685+0000 7f86d33c9700 1 -- 172.21.15.70:0/3002710775 wait complete. 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='osd.2 v1:172.21.15.70:6809/3606696222' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='osd.2 v1:172.21.15.70:6809/3606696222' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:37.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:37 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='osd.2 v1:172.21.15.70:6809/3606696222' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:38.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:38.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:38.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:37 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:35:38.361 DEBUG:teuthology.orchestra.run.smithi070:osd.2> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.2.service 2024-02-10T07:35:38.363 INFO:tasks.cephadm:Deploying osd.3 on smithi070 with /dev/vg_nvme/lv_1... 2024-02-10T07:35:38.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-10T07:35:38.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[29548]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T07:35:38.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[29548]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T07:35:38.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:38.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[29548]: from='osd.2 v1:172.21.15.70:6809/3606696222' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:38.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[29548]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:38.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[39923]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T07:35:38.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[39923]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T07:35:38.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:38.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[39923]: from='osd.2 v1:172.21.15.70:6809/3606696222' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:38.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-mon[39923]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:38.958 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2[55564]: 2024-02-10T07:35:38.750+0000 7fcc0f12a700 -1 osd.2 0 waiting for initial osdmap 2024-02-10T07:35:38.958 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:38 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2[55564]: 2024-02-10T07:35:38.753+0000 7fcc0af1d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:35:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:38 smithi155 ceph-mon[32688]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T07:35:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:38 smithi155 ceph-mon[32688]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T07:35:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:38 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:38 smithi155 ceph-mon[32688]: from='osd.2 v1:172.21.15.70:6809/3606696222' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:38 smithi155 ceph-mon[32688]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:35:39.584 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:35:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[29548]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:35:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[29548]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T07:35:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[29548]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[39923]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:35:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[39923]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T07:35:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:39 smithi070 ceph-mon[39923]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:39 smithi155 ceph-mon[32688]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:35:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:39 smithi155 ceph-mon[32688]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T07:35:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:39 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:39 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:39 smithi155 ceph-mon[32688]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:35:40.869 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:35:40.869 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[29548]: osd.2 v1:172.21.15.70:6809/3606696222 boot 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[29548]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[39923]: osd.2 v1:172.21.15.70:6809/3606696222 boot 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[39923]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T07:35:40.870 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:40 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:40 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:35:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:40 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:35:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:40 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:40 smithi155 ceph-mon[32688]: osd.2 v1:172.21.15.70:6809/3606696222 boot 2024-02-10T07:35:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:40 smithi155 ceph-mon[32688]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T07:35:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:40 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:35:41.833 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-10T07:35:41.833 INFO:teuthology.orchestra.run.smithi070.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-10T07:35:41.833 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10+0 records in 2024-02-10T07:35:41.833 INFO:teuthology.orchestra.run.smithi070.stderr:10+0 records out 2024-02-10T07:35:41.833 INFO:teuthology.orchestra.run.smithi070.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00868207 s, 1.2 GB/s 2024-02-10T07:35:41.833 INFO:teuthology.orchestra.run.smithi070.stderr:--> Zapping successful for: 2024-02-10T07:35:42.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:41 smithi070 ceph-mon[29548]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T07:35:42.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:41 smithi070 ceph-mon[29548]: pgmap v76: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:42.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:41 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:35:42.102 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:41 smithi070 ceph-mon[39923]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T07:35:42.102 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:41 smithi070 ceph-mon[39923]: pgmap v76: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:42.102 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:41 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:35:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:41 smithi155 ceph-mon[32688]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T07:35:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:41 smithi155 ceph-mon[32688]: pgmap v76: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:41 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:35:42.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi070:vg_nvme/lv_1 2024-02-10T07:35:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:35:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:35:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:35:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.617 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59790]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:35:43.617 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59790]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:43.618 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59790]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:43.618 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59790]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:43.962 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59810]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:35:43.962 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:43.962 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59810]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:43.963 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59810]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:43.963 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi070 2024-02-10T07:35:43.963 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi070 to 5354M 2024-02-10T07:35:43.963 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:35:43.963 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[29548]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T07:35:43.963 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[29548]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:43.963 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:43.963 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi070 2024-02-10T07:35:43.964 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi070 to 5354M 2024-02-10T07:35:43.964 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:35:43.964 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[39923]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T07:35:43.964 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[39923]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:43.964 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:43 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:44.218 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59815]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:35:44.218 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:44.218 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59815]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:44.218 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:43 smithi070 sudo[59815]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:43 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi070 2024-02-10T07:35:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:43 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi070 to 5354M 2024-02-10T07:35:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:43 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:35:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:43 smithi155 ceph-mon[32688]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T07:35:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:43 smithi155 ceph-mon[32688]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:43 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:44.473 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59818]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:35:44.473 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:44.473 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59818]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:44.473 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59818]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.735+0000 7ff9ebff3700 1 -- 172.21.15.70:0/2296593679 >> v1:172.21.15.70:6790/0 conn(0x7ff9e4074930 legacy=0x7ff9e410a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- 172.21.15.70:0/2296593679 shutdown_connections 2024-02-10T07:35:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- 172.21.15.70:0/2296593679 >> 172.21.15.70:0/2296593679 conn(0x7ff9e40fd120 msgr2=0x7ff9e40ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:35:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- 172.21.15.70:0/2296593679 shutdown_connections 2024-02-10T07:35:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- 172.21.15.70:0/2296593679 wait complete. 2024-02-10T07:35:44.737 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59841]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:35:44.737 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59841]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:44.737 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59841]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:44.737 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59841]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 Processor -- start 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- start start 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9e4199bf0 con 0x7ff9e4075cd0 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9e41a53e0 con 0x7ff9e410c0e0 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.736+0000 7ff9ebff3700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9e41a6580 con 0x7ff9e4074930 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9e958e700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff9e4075cd0 0x7ff9e419f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55384/0 (socket says 172.21.15.70:55384) 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9e958e700 1 -- 172.21.15.70:0/1036449622 learned_addr learned my addr 172.21.15.70:0/1036449622 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318933487 0 0) 0x7ff9e41a6580 con 0x7ff9e4074930 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d00035b0 con 0x7ff9e4074930 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781493402 0 0) 0x7ff9e4199bf0 con 0x7ff9e4075cd0 2024-02-10T07:35:44.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e41a6580 con 0x7ff9e4075cd0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828169262 0 0) 0x7ff9e41a53e0 con 0x7ff9e410c0e0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e4199bf0 con 0x7ff9e410c0e0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1098084162 0 0) 0x7ff9e41a6580 con 0x7ff9e4075cd0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9e41a53e0 con 0x7ff9e4075cd0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1296127363 0 0) 0x7ff9d00035b0 con 0x7ff9e4074930 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9e41a6580 con 0x7ff9e4074930 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff9d4004440 con 0x7ff9e4075cd0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1654837647 0 0) 0x7ff9e4199bf0 con 0x7ff9e410c0e0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9d00035b0 con 0x7ff9e410c0e0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff9e0001830 con 0x7ff9e4074930 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff9dc001b00 con 0x7ff9e410c0e0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 963135837 0 0) 0x7ff9e41a53e0 con 0x7ff9e4075cd0 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 >> v1:172.21.15.70:6790/0 conn(0x7ff9e4074930 legacy=0x7ff9e41990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:44.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.737+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 >> v1:172.21.15.155:6789/0 conn(0x7ff9e410c0e0 legacy=0x7ff9e41a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.738+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e41a7720 con 0x7ff9e4075cd0 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.738+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e41a4430 con 0x7ff9e4075cd0 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.738+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e41a4930 con 0x7ff9e4075cd0 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.738+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7ff9d4001af0 con 0x7ff9e4075cd0 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.738+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff9d4005050 con 0x7ff9e4075cd0 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.739+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c8005150 con 0x7ff9e4075cd0 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.739+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7ff9d40051b0 con 0x7ff9e4075cd0 2024-02-10T07:35:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.740+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(22..22 src has 1..22) v4 ==== 2511+0+0 (unknown 2861043223 0 0) 0x7ff9d4087c50 con 0x7ff9e4075cd0 2024-02-10T07:35:44.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.745+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7ff9d4055e00 con 0x7ff9e4075cd0 2024-02-10T07:35:44.955 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:35:44.955+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9c8000bc0 con 0x7ff9d006eac0 2024-02-10T07:35:44.991 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59853]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:35:44.991 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:44.991 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59853]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:44.991 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59853]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:45.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59874]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:35:45.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:45.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:44 smithi070 sudo[59874]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:45.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 sudo[59874]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 ceph-mon[32688]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 ceph-mon[32688]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35240]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35240]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:45.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[39923]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-10T07:35:45.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[39923]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T07:35:45.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:45.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:45.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 sudo[59877]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 sudo[59877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 sudo[59877]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 sudo[59877]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[29548]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[29548]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35240]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35243]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:35:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35243]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:45 smithi155 sudo[35243]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:46.105 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60007]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:35:46.105 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60007]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:46.105 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60007]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:46.105 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60007]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:46.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:46.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:46.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='client.14307 v1:172.21.15.70:0/1036449622' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:46.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60022]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60022]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60022]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 sudo[60022]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='client.14307 v1:172.21.15.70:0/1036449622' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:46.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:46.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='client.14307 v1:172.21.15.70:0/1036449622' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:35:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:35:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:35:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:35:47.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:47 smithi070 ceph-mon[29548]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:47.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:47 smithi070 ceph-mon[29548]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:47.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:47 smithi070 ceph-mon[29548]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:47.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:47 smithi070 ceph-mon[39923]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:47.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:47 smithi070 ceph-mon[39923]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:47.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:47 smithi070 ceph-mon[39923]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:47 smithi155 ceph-mon[32688]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:35:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:47 smithi155 ceph-mon[32688]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:35:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:47 smithi155 ceph-mon[32688]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4103691096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]: dispatch 2024-02-10T07:35:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[39923]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]: dispatch 2024-02-10T07:35:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[39923]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]': finished 2024-02-10T07:35:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[39923]: osdmap e23: 4 total, 3 up, 4 in 2024-02-10T07:35:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:35:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1610997375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:35:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4103691096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]: dispatch 2024-02-10T07:35:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[29548]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]: dispatch 2024-02-10T07:35:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[29548]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]': finished 2024-02-10T07:35:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[29548]: osdmap e23: 4 total, 3 up, 4 in 2024-02-10T07:35:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:35:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1610997375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:35:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4103691096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]: dispatch 2024-02-10T07:35:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:48 smithi155 ceph-mon[32688]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]: dispatch 2024-02-10T07:35:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:48 smithi155 ceph-mon[32688]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5953e938-5967-420e-9262-2c2ba2ab3bd1"}]': finished 2024-02-10T07:35:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:48 smithi155 ceph-mon[32688]: osdmap e23: 4 total, 3 up, 4 in 2024-02-10T07:35:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:48 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:35:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1610997375' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:35:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:49 smithi070 ceph-mon[29548]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:49 smithi070 ceph-mon[39923]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:49 smithi155 ceph-mon[32688]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:52.167 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:51 smithi070 ceph-mon[29548]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:52.168 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:51 smithi070 ceph-mon[39923]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:51 smithi155 ceph-mon[32688]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:35:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:53 smithi070 ceph-mon[29548]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-02-10T07:35:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:53 smithi070 ceph-mon[39923]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-02-10T07:35:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:53 smithi155 ceph-mon[32688]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-02-10T07:35:55.145 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T07:35:55.145 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:55.146 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T07:35:55.146 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T07:35:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:35:56.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:55 smithi070 ceph-mon[29548]: Deploying daemon osd.3 on smithi070 2024-02-10T07:35:56.186 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:55 smithi070 ceph-mon[29548]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-02-10T07:35:56.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:55 smithi070 ceph-mon[39923]: Deploying daemon osd.3 on smithi070 2024-02-10T07:35:56.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:55 smithi070 ceph-mon[39923]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-02-10T07:35:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:55 smithi155 ceph-mon[32688]: Deploying daemon osd.3 on smithi070 2024-02-10T07:35:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:55 smithi155 ceph-mon[32688]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-02-10T07:35:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:57 smithi070 ceph-mon[29548]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-10T07:35:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:57 smithi070 ceph-mon[39923]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-10T07:35:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:57 smithi155 ceph-mon[32688]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-02-10T07:36:00.198 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[39923]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 847 B/s rd, 24 KiB/s wr, 2 op/s 2024-02-10T07:36:00.198 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:00.198 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:00.198 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:00.199 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[29548]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 847 B/s rd, 24 KiB/s wr, 2 op/s 2024-02-10T07:36:00.199 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:00.199 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:00.199 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:35:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:59 smithi155 ceph-mon[32688]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 847 B/s rd, 24 KiB/s wr, 2 op/s 2024-02-10T07:36:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:35:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:02.163 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:01 smithi070 ceph-mon[29548]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T07:36:02.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:01 smithi070 ceph-mon[39923]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T07:36:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:01 smithi155 ceph-mon[32688]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T07:36:03.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:36:03.556+0000 7ff9daffd700 1 -- 172.21.15.70:0/1036449622 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3786370151) 0x7ff9c8000bc0 con 0x7ff9d006eac0 2024-02-10T07:36:03.558 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 3 on host 'smithi070' 2024-02-10T07:36:03.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:36:03.559+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 >> v1:172.21.15.70:6800/3858935040 conn(0x7ff9d006eac0 legacy=0x7ff9d0070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:36:03.559+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 >> v1:172.21.15.70:6789/0 conn(0x7ff9e4075cd0 legacy=0x7ff9e419f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:36:03.559+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 shutdown_connections 2024-02-10T07:36:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:36:03.559+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 >> 172.21.15.70:0/1036449622 conn(0x7ff9e40fd120 msgr2=0x7ff9e40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:36:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:36:03.559+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 shutdown_connections 2024-02-10T07:36:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:36:03.559+0000 7ff9ebff3700 1 -- 172.21.15.70:0/1036449622 wait complete. 2024-02-10T07:36:04.131 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T07:36:04.131 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.131 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.131 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:04.131 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='osd.3 v1:172.21.15.70:6813/3512826399' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:04.132 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.133 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='osd.3 v1:172.21.15.70:6813/3512826399' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T07:36:04.133 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T07:36:04.133 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.133 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.133 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:03 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:04.158 DEBUG:teuthology.orchestra.run.smithi070:osd.3> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.3.service 2024-02-10T07:36:04.160 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_4... 2024-02-10T07:36:04.160 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-10T07:36:04.189 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='osd.3 v1:172.21.15.70:6813/3512826399' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:04.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:03 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:04.404 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:36:05.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[39923]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T07:36:05.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[39923]: osdmap e24: 4 total, 3 up, 4 in 2024-02-10T07:36:05.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:05.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[39923]: from='osd.3 v1:172.21.15.70:6813/3512826399' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:36:05.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[39923]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:36:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[29548]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T07:36:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[29548]: osdmap e24: 4 total, 3 up, 4 in 2024-02-10T07:36:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[29548]: from='osd.3 v1:172.21.15.70:6813/3512826399' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:36:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-mon[29548]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:36:05.242 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3[61781]: 2024-02-10T07:36:04.881+0000 7f2aea969700 -1 osd.3 0 waiting for initial osdmap 2024-02-10T07:36:05.242 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 07:36:04 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3[61781]: 2024-02-10T07:36:04.885+0000 7f2ae675c700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:36:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:04 smithi155 ceph-mon[32688]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T07:36:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:04 smithi155 ceph-mon[32688]: osdmap e24: 4 total, 3 up, 4 in 2024-02-10T07:36:05.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:04 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:05.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:04 smithi155 ceph-mon[32688]: from='osd.3 v1:172.21.15.70:6813/3512826399' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:36:05.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:04 smithi155 ceph-mon[32688]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-02-10T07:36:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:05 smithi155 ceph-mon[32688]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:36:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:05 smithi155 ceph-mon[32688]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:36:06.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:05 smithi155 ceph-mon[32688]: osdmap e25: 4 total, 3 up, 4 in 2024-02-10T07:36:06.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:05 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:06.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:05 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:06.226 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-10T07:36:06.227 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-10T07:36:06.227 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-10T07:36:06.227 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-10T07:36:06.227 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124234 s, 844 MB/s 2024-02-10T07:36:06.227 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-10T07:36:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[29548]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:36:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[29548]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:36:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[29548]: osdmap e25: 4 total, 3 up, 4 in 2024-02-10T07:36:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[39923]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:36:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[39923]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-02-10T07:36:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[39923]: osdmap e25: 4 total, 3 up, 4 in 2024-02-10T07:36:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:05 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:06.884 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-02-10T07:36:07.061 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[39923]: osd.3 v1:172.21.15.70:6813/3512826399 boot 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[39923]: osdmap e26: 4 total, 4 up, 4 in 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[29548]: osd.3 v1:172.21.15.70:6813/3512826399 boot 2024-02-10T07:36:07.062 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[29548]: osdmap e26: 4 total, 4 up, 4 in 2024-02-10T07:36:07.063 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:07.160 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:36:07.225 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:06 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:36:07.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:06 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:36:07.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:07.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:06 smithi155 ceph-mon[32688]: osd.3 v1:172.21.15.70:6813/3512826399 boot 2024-02-10T07:36:07.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:06 smithi155 ceph-mon[32688]: osdmap e26: 4 total, 4 up, 4 in 2024-02-10T07:36:07.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:36:08.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:36:08.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: osdmap e27: 4 total, 4 up, 4 in 2024-02-10T07:36:08.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.223 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.223 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.223 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.223 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.223 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:08.223 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:08.223 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:36:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: osdmap e27: 4 total, 4 up, 4 in 2024-02-10T07:36:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: osdmap e27: 4 total, 4 up, 4 in 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:08.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.737+0000 7f81d4c94700 1 -- 172.21.15.155:0/3382589737 >> v1:172.21.15.155:6789/0 conn(0x7f81d0109ca0 legacy=0x7f81d010c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:08.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.738+0000 7f81d4c94700 1 -- 172.21.15.155:0/3382589737 shutdown_connections 2024-02-10T07:36:08.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.738+0000 7f81d4c94700 1 -- 172.21.15.155:0/3382589737 >> 172.21.15.155:0/3382589737 conn(0x7f81d00fc000 msgr2=0x7f81d00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:36:08.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.738+0000 7f81d4c94700 1 -- 172.21.15.155:0/3382589737 shutdown_connections 2024-02-10T07:36:08.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.738+0000 7f81d4c94700 1 -- 172.21.15.155:0/3382589737 wait complete. 2024-02-10T07:36:08.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.738+0000 7f81d4c94700 1 Processor -- start 2024-02-10T07:36:08.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.738+0000 7f81d4c94700 1 -- start start 2024-02-10T07:36:08.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81d4c94700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81d010ea10 con 0x7f81d0106100 2024-02-10T07:36:08.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81d4c94700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81d0197c20 con 0x7f81d01025f0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81ce59c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f81d01025f0 0x7f81d0197070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44416/0 (socket says 172.21.15.155:44416) 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81ce59c700 1 -- 172.21.15.155:0/1745220760 learned_addr learned my addr 172.21.15.155:0/1745220760 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81d4c94700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81d01a43c0 con 0x7f81d0109ca0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800379371 0 0) 0x7f81d0197c20 con 0x7f81d01025f0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b80035b0 con 0x7f81d01025f0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043792416 0 0) 0x7f81d010ea10 con 0x7f81d0106100 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d0197c20 con 0x7f81d0106100 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321414269 0 0) 0x7f81d01a43c0 con 0x7f81d0109ca0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d010ea10 con 0x7f81d0109ca0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1458482421 0 0) 0x7f81b80035b0 con 0x7f81d01025f0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81d01a43c0 con 0x7f81d01025f0 2024-02-10T07:36:08.741 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f81c00018b0 con 0x7f81d01025f0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 137535875 0 0) 0x7f81d010ea10 con 0x7f81d0109ca0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81b80035b0 con 0x7f81d0109ca0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1591466715 0 0) 0x7f81d0197c20 con 0x7f81d0106100 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81d010ea10 con 0x7f81d0106100 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.739+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f81c8001c00 con 0x7f81d0109ca0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f81c40017e0 con 0x7f81d0106100 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3018369799 0 0) 0x7f81d01a43c0 con 0x7f81d01025f0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 >> v1:172.21.15.70:6790/0 conn(0x7f81d0109ca0 legacy=0x7f81d01a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 >> v1:172.21.15.70:6789/0 conn(0x7f81d0106100 legacy=0x7f81d019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d01a5560 con 0x7f81d01025f0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d01a45b0 con 0x7f81d01025f0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d01a4ab0 con 0x7f81d01025f0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f81c0003b30 con 0x7f81d01025f0 2024-02-10T07:36:08.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.740+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f81c0005c30 con 0x7f81d01025f0 2024-02-10T07:36:08.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.741+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b0005150 con 0x7f81d01025f0 2024-02-10T07:36:08.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.741+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f81c00036a0 con 0x7f81d01025f0 2024-02-10T07:36:08.743 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.742+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(27..27 src has 1..27) v4 ==== 2803+0+0 (unknown 2633333550 0 0) 0x7f81c0087b90 con 0x7f81d01025f0 2024-02-10T07:36:08.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.746+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f81c0052050 con 0x7f81d01025f0 2024-02-10T07:36:08.949 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:08.949+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f81b0000bc0 con 0x7f81b806e920 2024-02-10T07:36:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:08 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi070 2024-02-10T07:36:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:08 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi070 to 4016M 2024-02-10T07:36:09.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:08 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi070 2024-02-10T07:36:09.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:08 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi070 to 4016M 2024-02-10T07:36:09.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:08 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi070 2024-02-10T07:36:09.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:08 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi070 to 4016M 2024-02-10T07:36:10.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:09 smithi155 ceph-mon[32688]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:10.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:09 smithi155 ceph-mon[32688]: from='client.24226 v1:172.21.15.155:0/1745220760' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:36:10.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:09 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:36:10.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:09 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:36:10.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:09 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[29548]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[29548]: from='client.24226 v1:172.21.15.155:0/1745220760' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:36:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:36:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:36:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[39923]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[39923]: from='client.24226 v1:172.21.15.155:0/1745220760' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:36:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:36:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:36:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:09 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:12.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:11 smithi155 ceph-mon[32688]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:12.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/4202587749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]: dispatch 2024-02-10T07:36:12.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:11 smithi155 ceph-mon[32688]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]: dispatch 2024-02-10T07:36:12.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:11 smithi155 ceph-mon[32688]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]': finished 2024-02-10T07:36:12.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:11 smithi155 ceph-mon[32688]: osdmap e28: 5 total, 4 up, 5 in 2024-02-10T07:36:12.162 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:11 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[29548]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/4202587749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]: dispatch 2024-02-10T07:36:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[29548]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]: dispatch 2024-02-10T07:36:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[29548]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]': finished 2024-02-10T07:36:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[29548]: osdmap e28: 5 total, 4 up, 5 in 2024-02-10T07:36:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[39923]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/4202587749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]: dispatch 2024-02-10T07:36:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[39923]: from='client.24232 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]: dispatch 2024-02-10T07:36:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[39923]: from='client.24232 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51"}]': finished 2024-02-10T07:36:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[39923]: osdmap e28: 5 total, 4 up, 5 in 2024-02-10T07:36:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:11 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:13.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/2231954896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:36:13.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/2231954896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:36:13.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/2231954896' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:36:14.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:13 smithi070 ceph-mon[29548]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:14.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:13 smithi070 ceph-mon[39923]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:13 smithi155 ceph-mon[32688]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:15 smithi070 ceph-mon[29548]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:16.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:15 smithi070 ceph-mon[39923]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:15 smithi155 ceph-mon[32688]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:17 smithi070 ceph-mon[29548]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:17 smithi070 ceph-mon[39923]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:17 smithi155 ceph-mon[32688]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:18.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T07:36:18.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T07:36:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:18.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T07:36:18.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:19.819 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:19 smithi155 ceph-mon[32688]: Deploying daemon osd.4 on smithi155 2024-02-10T07:36:19.819 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:19 smithi155 ceph-mon[32688]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:19 smithi070 ceph-mon[39923]: Deploying daemon osd.4 on smithi155 2024-02-10T07:36:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:19 smithi070 ceph-mon[39923]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:19 smithi070 ceph-mon[29548]: Deploying daemon osd.4 on smithi155 2024-02-10T07:36:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:19 smithi070 ceph-mon[29548]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:22.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:21 smithi155 ceph-mon[32688]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:22.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:21 smithi070 ceph-mon[29548]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:22.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:21 smithi070 ceph-mon[39923]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:23 smithi155 ceph-mon[32688]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:24.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[29548]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:24.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:24.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:24.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:24.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[39923]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:24.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:24.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:24.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:26.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:25 smithi070 ceph-mon[29548]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:26.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:25 smithi070 ceph-mon[39923]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:25 smithi155 ceph-mon[32688]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:26.710 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:26.709+0000 7f81bf7fe700 1 -- 172.21.15.155:0/1745220760 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3435391443) 0x7f81b0000bc0 con 0x7f81b806e920 2024-02-10T07:36:26.710 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2024-02-10T07:36:26.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:26.712+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 >> v1:172.21.15.70:6800/3858935040 conn(0x7f81b806e920 legacy=0x7f81b8070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:26.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:26.712+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 >> v1:172.21.15.155:6789/0 conn(0x7f81d01025f0 legacy=0x7f81d0197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:26.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:26.712+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 shutdown_connections 2024-02-10T07:36:26.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:26.712+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 >> 172.21.15.155:0/1745220760 conn(0x7f81d00fc000 msgr2=0x7f81d00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:36:26.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:26.712+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 shutdown_connections 2024-02-10T07:36:26.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:26.712+0000 7f81d4c94700 1 -- 172.21.15.155:0/1745220760 wait complete. 2024-02-10T07:36:27.340 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='osd.4 v1:172.21.15.155:6800/1901321685' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.341 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:27 smithi155 ceph-mon[32688]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:27.364 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.4.service 2024-02-10T07:36:27.366 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-02-10T07:36:27.366 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-10T07:36:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='osd.4 v1:172.21.15.155:6800/1901321685' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[29548]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='osd.4 v1:172.21.15.155:6800/1901321685' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T07:36:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T07:36:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:27 smithi070 ceph-mon[39923]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:28.532 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:36:28.620 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:28 smithi155 ceph-mon[32688]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T07:36:28.620 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:28 smithi155 ceph-mon[32688]: osdmap e29: 5 total, 4 up, 5 in 2024-02-10T07:36:28.620 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:28.620 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:28 smithi155 ceph-mon[32688]: from='osd.4 v1:172.21.15.155:6800/1901321685' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:28.620 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:28 smithi155 ceph-mon[32688]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:28.621 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 07:36:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T07:36:28.338+0000 7f4980f55700 -1 osd.4 0 waiting for initial osdmap 2024-02-10T07:36:28.621 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 07:36:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T07:36:28.342+0000 7f497cd48700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:36:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[29548]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T07:36:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[29548]: osdmap e29: 5 total, 4 up, 5 in 2024-02-10T07:36:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[29548]: from='osd.4 v1:172.21.15.155:6800/1901321685' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[29548]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[39923]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T07:36:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[39923]: osdmap e29: 5 total, 4 up, 5 in 2024-02-10T07:36:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[39923]: from='osd.4 v1:172.21.15.155:6800/1901321685' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:28 smithi070 ceph-mon[39923]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:29 smithi155 ceph-mon[32688]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:36:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:29 smithi155 ceph-mon[32688]: osdmap e30: 5 total, 4 up, 5 in 2024-02-10T07:36:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:29.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:29.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:29 smithi155 ceph-mon[32688]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:29.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:29 smithi155 ceph-mon[32688]: osd.4 v1:172.21.15.155:6800/1901321685 boot 2024-02-10T07:36:29.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[29548]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:36:29.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[29548]: osdmap e30: 5 total, 4 up, 5 in 2024-02-10T07:36:29.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:29.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:29.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[29548]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:29.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[29548]: osd.4 v1:172.21.15.155:6800/1901321685 boot 2024-02-10T07:36:29.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[39923]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:36:29.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[39923]: osdmap e30: 5 total, 4 up, 5 in 2024-02-10T07:36:29.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:29.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:29.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[39923]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-02-10T07:36:29.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:29 smithi070 ceph-mon[39923]: osd.4 v1:172.21.15.155:6800/1901321685 boot 2024-02-10T07:36:30.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:36:30.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:36:30.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[39923]: osdmap e31: 5 total, 5 up, 5 in 2024-02-10T07:36:30.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:30.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[39923]: osdmap e32: 5 total, 5 up, 5 in 2024-02-10T07:36:30.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:36:30.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:36:30.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[29548]: osdmap e31: 5 total, 5 up, 5 in 2024-02-10T07:36:30.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:30.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:30 smithi070 ceph-mon[29548]: osdmap e32: 5 total, 5 up, 5 in 2024-02-10T07:36:30.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:30 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:36:30.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:30 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:36:30.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:30 smithi155 ceph-mon[32688]: osdmap e31: 5 total, 5 up, 5 in 2024-02-10T07:36:30.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:30 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:36:30.542 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:30 smithi155 ceph-mon[32688]: osdmap e32: 5 total, 5 up, 5 in 2024-02-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-10T07:36:31.351 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138802 s, 755 MB/s 2024-02-10T07:36:31.352 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-10T07:36:31.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi155 2024-02-10T07:36:31.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi155 to 17088M 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:31.949 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:31 smithi155 ceph-mon[32688]: osdmap e33: 5 total, 5 up, 5 in 2024-02-10T07:36:31.975 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-02-10T07:36:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi155 2024-02-10T07:36:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi155 to 17088M 2024-02-10T07:36:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[39923]: osdmap e33: 5 total, 5 up, 5 in 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi155 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi155 to 17088M 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:31.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:31.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:31.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:31.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:31 smithi070 ceph-mon[29548]: osdmap e33: 5 total, 5 up, 5 in 2024-02-10T07:36:32.207 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:36:32.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:32 smithi070 ceph-mon[29548]: osdmap e34: 5 total, 5 up, 5 in 2024-02-10T07:36:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:32 smithi070 ceph-mon[39923]: osdmap e34: 5 total, 5 up, 5 in 2024-02-10T07:36:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:32 smithi155 ceph-mon[32688]: osdmap e34: 5 total, 5 up, 5 in 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- 172.21.15.155:0/2337286494 >> v1:172.21.15.155:6789/0 conn(0x7f05ec104600 legacy=0x7f05ec106a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- 172.21.15.155:0/2337286494 shutdown_connections 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- 172.21.15.155:0/2337286494 >> 172.21.15.155:0/2337286494 conn(0x7f05ec0fb790 msgr2=0x7f05ec0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- 172.21.15.155:0/2337286494 shutdown_connections 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- 172.21.15.155:0/2337286494 wait complete. 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 Processor -- start 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- start start 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f05ec1a4140 con 0x7f05ec1001d0 2024-02-10T07:36:33.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f05ec1a5340 con 0x7f05ec104600 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.817+0000 7f05f33db700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f05ec1a6540 con 0x7f05ec108340 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05f0976700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f05ec104600 0x7f05ec19ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:46104/0 (socket says 172.21.15.155:46104) 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05f0976700 1 -- 172.21.15.155:0/1728544755 learned_addr learned my addr 172.21.15.155:0/1728544755 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507766176 0 0) 0x7f05ec1a5340 con 0x7f05ec104600 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d80046c0 con 0x7f05ec104600 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760236283 0 0) 0x7f05ec1a4140 con 0x7f05ec1001d0 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec1a5340 con 0x7f05ec1001d0 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228653319 0 0) 0x7f05ec1a6540 con 0x7f05ec108340 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec1a4140 con 0x7f05ec108340 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1338048177 0 0) 0x7f05d80046c0 con 0x7f05ec104600 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05ec1a6540 con 0x7f05ec104600 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f05dc001430 con 0x7f05ec104600 2024-02-10T07:36:33.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4021821503 0 0) 0x7f05ec1a4140 con 0x7f05ec108340 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05d80046c0 con 0x7f05ec108340 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f05e4001ae0 con 0x7f05ec108340 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 539625499 0 0) 0x7f05ec1a5340 con 0x7f05ec1001d0 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05ec1a4140 con 0x7f05ec1001d0 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f05e8001830 con 0x7f05ec1001d0 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3673864764 0 0) 0x7f05ec1a6540 con 0x7f05ec104600 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 >> v1:172.21.15.70:6790/0 conn(0x7f05ec108340 legacy=0x7f05ec1a2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 >> v1:172.21.15.70:6789/0 conn(0x7f05ec1001d0 legacy=0x7f05ec1064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec1a7740 con 0x7f05ec104600 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec1a4330 con 0x7f05ec104600 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec1a4830 con 0x7f05ec104600 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f05dc0016b0 con 0x7f05ec104600 2024-02-10T07:36:33.822 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.818+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f05dc004fe0 con 0x7f05ec104600 2024-02-10T07:36:33.823 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.819+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d0005120 con 0x7f05ec104600 2024-02-10T07:36:33.823 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.820+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f05dc005220 con 0x7f05ec104600 2024-02-10T07:36:33.823 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.820+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 3205+0+0 (unknown 1948348987 0 0) 0x7f05dc0880d0 con 0x7f05ec104600 2024-02-10T07:36:33.826 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:33.825+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f05dc0523a0 con 0x7f05ec104600 2024-02-10T07:36:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:33 smithi070 ceph-mon[29548]: osdmap e35: 5 total, 5 up, 5 in 2024-02-10T07:36:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:33 smithi070 ceph-mon[29548]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:33 smithi070 ceph-mon[39923]: osdmap e35: 5 total, 5 up, 5 in 2024-02-10T07:36:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:33 smithi070 ceph-mon[39923]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:33 smithi155 ceph-mon[32688]: osdmap e35: 5 total, 5 up, 5 in 2024-02-10T07:36:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:33 smithi155 ceph-mon[32688]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:34.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:34.027+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f05d0000b90 con 0x7f05d806e9b0 2024-02-10T07:36:34.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:34 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:36:34.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:34 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:36:34.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:34 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:34 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:36:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:34 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:36:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:34 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:34 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:36:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:34 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:36:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:34 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:35 smithi070 ceph-mon[29548]: from='client.24253 v1:172.21.15.155:0/1728544755' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:36:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:35 smithi070 ceph-mon[29548]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:35 smithi070 ceph-mon[39923]: from='client.24253 v1:172.21.15.155:0/1728544755' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:36:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:35 smithi070 ceph-mon[39923]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:35 smithi155 ceph-mon[32688]: from='client.24253 v1:172.21.15.155:0/1728544755' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:36:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:35 smithi155 ceph-mon[32688]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/3778487265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]: dispatch 2024-02-10T07:36:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[29548]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]: dispatch 2024-02-10T07:36:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[29548]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]': finished 2024-02-10T07:36:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[29548]: osdmap e36: 6 total, 5 up, 6 in 2024-02-10T07:36:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/3778487265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]: dispatch 2024-02-10T07:36:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[39923]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]: dispatch 2024-02-10T07:36:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[39923]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]': finished 2024-02-10T07:36:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[39923]: osdmap e36: 6 total, 5 up, 6 in 2024-02-10T07:36:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:36 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/3778487265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]: dispatch 2024-02-10T07:36:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:36 smithi155 ceph-mon[32688]: from='client.24259 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]: dispatch 2024-02-10T07:36:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:36 smithi155 ceph-mon[32688]: from='client.24259 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f131619-da44-4168-a436-8a7525448564"}]': finished 2024-02-10T07:36:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:36 smithi155 ceph-mon[32688]: osdmap e36: 6 total, 5 up, 6 in 2024-02-10T07:36:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:36 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:37 smithi070 ceph-mon[29548]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-10T07:36:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/998531034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:36:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:37 smithi070 ceph-mon[39923]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-10T07:36:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/998531034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:36:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:37 smithi155 ceph-mon[32688]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-10T07:36:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/998531034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:36:40.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:39 smithi155 ceph-mon[32688]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-10T07:36:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:39 smithi070 ceph-mon[29548]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-10T07:36:40.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:39 smithi070 ceph-mon[39923]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-10T07:36:42.226 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:41 smithi155 ceph-mon[32688]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-10T07:36:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:41 smithi070 ceph-mon[29548]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-10T07:36:42.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:41 smithi070 ceph-mon[39923]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-10T07:36:44.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:43 smithi155 ceph-mon[32688]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:36:44.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:43 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T07:36:44.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:43 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:44.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:43 smithi155 ceph-mon[32688]: Deploying daemon osd.5 on smithi155 2024-02-10T07:36:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[29548]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:36:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T07:36:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[29548]: Deploying daemon osd.5 on smithi155 2024-02-10T07:36:44.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[39923]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:36:44.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T07:36:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:43 smithi070 ceph-mon[39923]: Deploying daemon osd.5 on smithi155 2024-02-10T07:36:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:45 smithi155 ceph-mon[32688]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:36:46.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:45 smithi070 ceph-mon[29548]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:36:46.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:45 smithi070 ceph-mon[39923]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:36:47.730 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:47 smithi155 ceph-mon[32688]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:47 smithi070 ceph-mon[29548]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:47 smithi070 ceph-mon[39923]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:49.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:49 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:49.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:49 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:49.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:49 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:49.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:49 smithi155 ceph-mon[32688]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:49.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:49.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:49.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[39923]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:49 smithi070 ceph-mon[29548]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:51.787 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-02-10T07:36:51.787 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:51.786+0000 7f05e27fc700 1 -- 172.21.15.155:0/1728544755 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2429945703) 0x7f05d0000b90 con 0x7f05d806e9b0 2024-02-10T07:36:51.789 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:51.788+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 >> v1:172.21.15.70:6800/3858935040 conn(0x7f05d806e9b0 legacy=0x7f05d8070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:51.790 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:51.788+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 >> v1:172.21.15.155:6789/0 conn(0x7f05ec104600 legacy=0x7f05ec19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:51.790 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:51.788+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 shutdown_connections 2024-02-10T07:36:51.790 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:51.788+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 >> 172.21.15.155:0/1728544755 conn(0x7f05ec0fb790 msgr2=0x7f05ec107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:36:51.790 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:51.788+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 shutdown_connections 2024-02-10T07:36:51.790 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:51.788+0000 7f05f33db700 1 -- 172.21.15.155:0/1728544755 wait complete. 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='osd.5 v1:172.21.15.155:6804/969979851' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T07:36:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T07:36:52.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:51 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:52.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:52.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:52.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='osd.5 v1:172.21.15.155:6804/969979851' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='osd.5 v1:172.21.15.155:6804/969979851' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T07:36:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T07:36:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:51 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:36:52.448 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.5.service 2024-02-10T07:36:52.450 INFO:tasks.cephadm:Deploying osd.6 on smithi155 with /dev/vg_nvme/lv_2... 2024-02-10T07:36:52.451 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-10T07:36:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:52 smithi155 ceph-mon[32688]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T07:36:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:52 smithi155 ceph-mon[32688]: osdmap e37: 6 total, 5 up, 6 in 2024-02-10T07:36:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:52 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:52 smithi155 ceph-mon[32688]: from='osd.5 v1:172.21.15.155:6804/969979851' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:52 smithi155 ceph-mon[32688]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[29548]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T07:36:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[29548]: osdmap e37: 6 total, 5 up, 6 in 2024-02-10T07:36:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[29548]: from='osd.5 v1:172.21.15.155:6804/969979851' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[29548]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:53.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[39923]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T07:36:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[39923]: osdmap e37: 6 total, 5 up, 6 in 2024-02-10T07:36:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[39923]: from='osd.5 v1:172.21.15.155:6804/969979851' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:52 smithi070 ceph-mon[39923]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:36:53.543 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:36:53.639 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T07:36:53.378+0000 7fb1397dc700 -1 osd.5 0 waiting for initial osdmap 2024-02-10T07:36:53.639 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T07:36:53.382+0000 7fb1315c7700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:36:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[29548]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[29548]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:36:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[29548]: osdmap e38: 6 total, 5 up, 6 in 2024-02-10T07:36:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[29548]: from='osd.5 ' entity='osd.5' 2024-02-10T07:36:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[39923]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[39923]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:36:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[39923]: osdmap e38: 6 total, 5 up, 6 in 2024-02-10T07:36:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:53 smithi070 ceph-mon[39923]: from='osd.5 ' entity='osd.5' 2024-02-10T07:36:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-mon[32688]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-02-10T07:36:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-mon[32688]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:36:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-mon[32688]: osdmap e38: 6 total, 5 up, 6 in 2024-02-10T07:36:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:53 smithi155 ceph-mon[32688]: from='osd.5 ' entity='osd.5' 2024-02-10T07:36:55.211 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:54 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:36:55.211 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:54 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:36:55.211 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:54 smithi155 ceph-mon[32688]: osd.5 v1:172.21.15.155:6804/969979851 boot 2024-02-10T07:36:55.211 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:54 smithi155 ceph-mon[32688]: osdmap e39: 6 total, 6 up, 6 in 2024-02-10T07:36:55.211 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:54 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:36:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:36:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[29548]: osd.5 v1:172.21.15.155:6804/969979851 boot 2024-02-10T07:36:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[29548]: osdmap e39: 6 total, 6 up, 6 in 2024-02-10T07:36:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:36:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:36:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[39923]: osd.5 v1:172.21.15.155:6804/969979851 boot 2024-02-10T07:36:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[39923]: osdmap e39: 6 total, 6 up, 6 in 2024-02-10T07:36:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:54 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:36:56.132 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-10T07:36:56.132 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-10T07:36:56.133 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-10T07:36:56.133 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-10T07:36:56.133 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012446 s, 843 MB/s 2024-02-10T07:36:56.133 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-10T07:36:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:55 smithi070 ceph-mon[29548]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:36:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:55 smithi070 ceph-mon[29548]: osdmap e40: 6 total, 6 up, 6 in 2024-02-10T07:36:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:55 smithi070 ceph-mon[39923]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:36:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:55 smithi070 ceph-mon[39923]: osdmap e40: 6 total, 6 up, 6 in 2024-02-10T07:36:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:55 smithi155 ceph-mon[32688]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:36:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:55 smithi155 ceph-mon[32688]: osdmap e40: 6 total, 6 up, 6 in 2024-02-10T07:36:56.905 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-02-10T07:36:57.145 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:36:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: osdmap e41: 6 total, 6 up, 6 in 2024-02-10T07:36:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[29548]: osdmap e42: 6 total, 6 up, 6 in 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: osdmap e41: 6 total, 6 up, 6 in 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:56 smithi070 ceph-mon[39923]: osdmap e42: 6 total, 6 up, 6 in 2024-02-10T07:36:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: osdmap e41: 6 total, 6 up, 6 in 2024-02-10T07:36:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:36:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:36:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:36:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:36:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:56 smithi155 ceph-mon[32688]: osdmap e42: 6 total, 6 up, 6 in 2024-02-10T07:36:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi155 2024-02-10T07:36:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi155 to 8544M 2024-02-10T07:36:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[29548]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:36:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[29548]: osdmap e43: 6 total, 6 up, 6 in 2024-02-10T07:36:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[29548]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-10T07:36:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi155 2024-02-10T07:36:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi155 to 8544M 2024-02-10T07:36:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[39923]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:36:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[39923]: osdmap e43: 6 total, 6 up, 6 in 2024-02-10T07:36:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:57 smithi070 ceph-mon[39923]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-10T07:36:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:57 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi155 2024-02-10T07:36:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:57 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi155 to 8544M 2024-02-10T07:36:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:57 smithi155 ceph-mon[32688]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:36:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:57 smithi155 ceph-mon[32688]: osdmap e43: 6 total, 6 up, 6 in 2024-02-10T07:36:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:57 smithi155 ceph-mon[32688]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-10T07:36:58.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.768+0000 7f597e59e700 1 -- 172.21.15.155:0/2904653879 >> v1:172.21.15.155:6789/0 conn(0x7f5978106020 legacy=0x7f5978108410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:58.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.768+0000 7f597e59e700 1 -- 172.21.15.155:0/2904653879 shutdown_connections 2024-02-10T07:36:58.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.768+0000 7f597e59e700 1 -- 172.21.15.155:0/2904653879 >> 172.21.15.155:0/2904653879 conn(0x7f59780fb790 msgr2=0x7f59780fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:36:58.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.768+0000 7f597e59e700 1 -- 172.21.15.155:0/2904653879 shutdown_connections 2024-02-10T07:36:58.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.768+0000 7f597e59e700 1 -- 172.21.15.155:0/2904653879 wait complete. 2024-02-10T07:36:58.768 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.768+0000 7f597e59e700 1 Processor -- start 2024-02-10T07:36:58.769 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.768+0000 7f597e59e700 1 -- start start 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f597e59e700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5978197a30 con 0x7f59780ffe10 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f597e59e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59781a3220 con 0x7f5978109c80 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f597e59e700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59781a43c0 con 0x7f5978106020 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f597cb3b700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5978109c80 0x7f59781a0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:41368/0 (socket says 172.21.15.155:41368) 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f597cb3b700 1 -- 172.21.15.155:0/181305107 learned_addr learned my addr 172.21.15.155:0/181305107 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680257118 0 0) 0x7f59781a3220 con 0x7f5978109c80 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59600035b0 con 0x7f5978109c80 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708701213 0 0) 0x7f5978197a30 con 0x7f59780ffe10 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59781a3220 con 0x7f59780ffe10 2024-02-10T07:36:58.771 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606321766 0 0) 0x7f59781a43c0 con 0x7f5978106020 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5978197a30 con 0x7f5978106020 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4176376366 0 0) 0x7f59600035b0 con 0x7f5978109c80 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59781a43c0 con 0x7f5978109c80 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f5970001c30 con 0x7f5978109c80 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3825013362 0 0) 0x7f59781a3220 con 0x7f59780ffe10 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59600035b0 con 0x7f59780ffe10 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f59680017e0 con 0x7f59780ffe10 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 438572576 0 0) 0x7f5978197a30 con 0x7f5978106020 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59781a3220 con 0x7f5978106020 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f596c0044b0 con 0x7f5978106020 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2105858141 0 0) 0x7f59781a43c0 con 0x7f5978109c80 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 >> v1:172.21.15.70:6790/0 conn(0x7f5978106020 legacy=0x7f597819d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 >> v1:172.21.15.70:6789/0 conn(0x7f59780ffe10 legacy=0x7f5978196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:36:58.772 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.769+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59781a5560 con 0x7f5978109c80 2024-02-10T07:36:58.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.770+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59781a2270 con 0x7f5978109c80 2024-02-10T07:36:58.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.770+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f59781a2770 con 0x7f5978109c80 2024-02-10T07:36:58.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.770+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f5970001fc0 con 0x7f5978109c80 2024-02-10T07:36:58.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.770+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f5970005fa0 con 0x7f5978109c80 2024-02-10T07:36:58.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.770+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5964005150 con 0x7f5978109c80 2024-02-10T07:36:58.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.771+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f5970003cd0 con 0x7f5978109c80 2024-02-10T07:36:58.773 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.771+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(43..43 src has 1..43) v4 ==== 3497+0+0 (unknown 219336962 0 0) 0x7f59700880a0 con 0x7f5978109c80 2024-02-10T07:36:58.775 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.776+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f59700522b0 con 0x7f5978109c80 2024-02-10T07:36:58.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:36:58.978+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5964000bc0 con 0x7f596006e940 2024-02-10T07:37:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[29548]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[29548]: from='client.24280 v1:172.21.15.155:0/181305107' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:37:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:37:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[39923]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[39923]: from='client.24280 v1:172.21.15.155:0/181305107' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:37:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:37:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:36:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:59 smithi155 ceph-mon[32688]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:59 smithi155 ceph-mon[32688]: from='client.24280 v1:172.21.15.155:0/181305107' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:37:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:37:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:36:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:01 smithi155 ceph-mon[32688]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-10T07:37:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:01 smithi155 ceph-mon[32688]: Cluster is now healthy 2024-02-10T07:37:01.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:01 smithi070 ceph-mon[39923]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-10T07:37:01.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:01 smithi070 ceph-mon[39923]: Cluster is now healthy 2024-02-10T07:37:01.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:01 smithi070 ceph-mon[29548]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-10T07:37:01.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:01 smithi070 ceph-mon[29548]: Cluster is now healthy 2024-02-10T07:37:02.369 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:02 smithi155 ceph-mon[32688]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:02.369 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/1227146051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]: dispatch 2024-02-10T07:37:02.369 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:02 smithi155 ceph-mon[32688]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]: dispatch 2024-02-10T07:37:02.369 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:02 smithi155 ceph-mon[32688]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]': finished 2024-02-10T07:37:02.369 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:02 smithi155 ceph-mon[32688]: osdmap e44: 7 total, 6 up, 7 in 2024-02-10T07:37:02.369 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:02 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[29548]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/1227146051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]: dispatch 2024-02-10T07:37:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[29548]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]: dispatch 2024-02-10T07:37:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[29548]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]': finished 2024-02-10T07:37:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[29548]: osdmap e44: 7 total, 6 up, 7 in 2024-02-10T07:37:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[39923]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/1227146051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]: dispatch 2024-02-10T07:37:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[39923]: from='client.24286 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]: dispatch 2024-02-10T07:37:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[39923]: from='client.24286 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d2ab766-9537-43a1-8f08-621786e62715"}]': finished 2024-02-10T07:37:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[39923]: osdmap e44: 7 total, 6 up, 7 in 2024-02-10T07:37:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:02 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:03.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/1874130885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:37:03.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/1874130885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:37:03.504 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/1874130885' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:37:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:04 smithi070 ceph-mon[29548]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-10T07:37:04.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:04 smithi070 ceph-mon[39923]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-10T07:37:04.504 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:04 smithi155 ceph-mon[32688]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-10T07:37:06.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:06 smithi155 ceph-mon[32688]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-10T07:37:06.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:06 smithi070 ceph-mon[29548]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-10T07:37:06.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:06 smithi070 ceph-mon[39923]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-10T07:37:07.956 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:07 smithi155 ceph-mon[32688]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-10T07:37:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:07 smithi070 ceph-mon[29548]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-10T07:37:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:07 smithi070 ceph-mon[39923]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-10T07:37:08.901 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:08 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T07:37:08.901 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:08 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:08.901 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:08 smithi155 ceph-mon[32688]: Deploying daemon osd.6 on smithi155 2024-02-10T07:37:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:08 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T07:37:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:08 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:08 smithi070 ceph-mon[39923]: Deploying daemon osd.6 on smithi155 2024-02-10T07:37:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:08 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T07:37:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:08 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:08 smithi070 ceph-mon[29548]: Deploying daemon osd.6 on smithi155 2024-02-10T07:37:09.880 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:09 smithi155 ceph-mon[32688]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:09 smithi070 ceph-mon[29548]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:09 smithi070 ceph-mon[39923]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:12.240 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:11 smithi155 ceph-mon[32688]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:11 smithi070 ceph-mon[29548]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:12.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:11 smithi070 ceph-mon[39923]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:14.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:13 smithi155 ceph-mon[32688]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:14.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:13 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:14.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:13 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:14.158 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:13 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:14.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[39923]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:14.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:14.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:14.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:14.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[29548]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:14.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:14.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:14.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:13 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:15 smithi070 ceph-mon[29548]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:16.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:15 smithi070 ceph-mon[39923]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:15 smithi155 ceph-mon[32688]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:17.091 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 6 on host 'smithi155' 2024-02-10T07:37:17.092 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:17.091+0000 7f59757fa700 1 -- 172.21.15.155:0/181305107 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1961374907) 0x7f5964000bc0 con 0x7f596006e940 2024-02-10T07:37:17.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:17.094+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 >> v1:172.21.15.70:6800/3858935040 conn(0x7f596006e940 legacy=0x7f5960070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:17.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:17.094+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 >> v1:172.21.15.155:6789/0 conn(0x7f5978109c80 legacy=0x7f59781a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:17.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:17.094+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 shutdown_connections 2024-02-10T07:37:17.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:17.094+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 >> 172.21.15.155:0/181305107 conn(0x7f59780fb790 msgr2=0x7f59780fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:17.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:17.094+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 shutdown_connections 2024-02-10T07:37:17.094 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:17.094+0000 7f597e59e700 1 -- 172.21.15.155:0/181305107 wait complete. 2024-02-10T07:37:17.389 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:17 smithi155 ceph-mon[32688]: from='osd.6 v1:172.21.15.155:6808/3655535506' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T07:37:17.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:17 smithi155 ceph-mon[32688]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T07:37:17.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:17 smithi155 ceph-mon[32688]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:17.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:17 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[39923]: from='osd.6 v1:172.21.15.155:6808/3655535506' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T07:37:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[39923]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T07:37:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[39923]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[29548]: from='osd.6 v1:172.21.15.155:6808/3655535506' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T07:37:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[29548]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T07:37:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[29548]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:17 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:17.666 DEBUG:teuthology.orchestra.run.smithi155:osd.6> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.6.service 2024-02-10T07:37:17.668 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_1... 2024-02-10T07:37:17.668 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-10T07:37:17.961 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:37:18.439 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:18 smithi155 ceph-mon[32688]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T07:37:18.439 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:18 smithi155 ceph-mon[32688]: from='osd.6 v1:172.21.15.155:6808/3655535506' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:18.439 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:18 smithi155 ceph-mon[32688]: osdmap e45: 7 total, 6 up, 7 in 2024-02-10T07:37:18.439 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:18 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:18.440 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:18 smithi155 ceph-mon[32688]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:18.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[29548]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T07:37:18.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[29548]: from='osd.6 v1:172.21.15.155:6808/3655535506' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:18.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[29548]: osdmap e45: 7 total, 6 up, 7 in 2024-02-10T07:37:18.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:18.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[29548]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:18.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[39923]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T07:37:18.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[39923]: from='osd.6 v1:172.21.15.155:6808/3655535506' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:18.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[39923]: osdmap e45: 7 total, 6 up, 7 in 2024-02-10T07:37:18.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:18.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:18 smithi070 ceph-mon[39923]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:19.005 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 07:37:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T07:37:18.694+0000 7fac57cfa700 -1 osd.6 0 waiting for initial osdmap 2024-02-10T07:37:19.006 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 07:37:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T07:37:18.700+0000 7fac4fae5700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:37:19.299 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:19 smithi155 ceph-mon[32688]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:37:19.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:19 smithi155 ceph-mon[32688]: osdmap e46: 7 total, 6 up, 7 in 2024-02-10T07:37:19.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:19 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:19.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:19 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:19.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:19 smithi155 ceph-mon[32688]: from='osd.6 ' entity='osd.6' 2024-02-10T07:37:19.300 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:19 smithi155 ceph-mon[32688]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[29548]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:37:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[29548]: osdmap e46: 7 total, 6 up, 7 in 2024-02-10T07:37:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[29548]: from='osd.6 ' entity='osd.6' 2024-02-10T07:37:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[29548]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[39923]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:37:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[39923]: osdmap e46: 7 total, 6 up, 7 in 2024-02-10T07:37:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[39923]: from='osd.6 ' entity='osd.6' 2024-02-10T07:37:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:19 smithi070 ceph-mon[39923]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-02-10T07:37:20.314 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:20 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:37:20.314 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:20 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:37:20.314 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:20 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:20.314 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:20 smithi155 ceph-mon[32688]: osd.6 v1:172.21.15.155:6808/3655535506 boot 2024-02-10T07:37:20.314 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:20 smithi155 ceph-mon[32688]: osdmap e47: 7 total, 7 up, 7 in 2024-02-10T07:37:20.314 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:20 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:37:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:37:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[29548]: osd.6 v1:172.21.15.155:6808/3655535506 boot 2024-02-10T07:37:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[29548]: osdmap e47: 7 total, 7 up, 7 in 2024-02-10T07:37:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:37:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:37:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[39923]: osd.6 v1:172.21.15.155:6808/3655535506 boot 2024-02-10T07:37:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[39923]: osdmap e47: 7 total, 7 up, 7 in 2024-02-10T07:37:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:20 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:37:21.105 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-10T07:37:21.105 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-10T07:37:21.105 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-10T07:37:21.105 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-10T07:37:21.105 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135644 s, 773 MB/s 2024-02-10T07:37:21.105 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 2024-02-10T07:37:21.106 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-10T07:37:21.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:21 smithi155 ceph-mon[32688]: osdmap e48: 7 total, 7 up, 7 in 2024-02-10T07:37:21.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:21 smithi155 ceph-mon[32688]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:21 smithi070 ceph-mon[39923]: osdmap e48: 7 total, 7 up, 7 in 2024-02-10T07:37:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:21 smithi070 ceph-mon[39923]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:21 smithi070 ceph-mon[29548]: osdmap e48: 7 total, 7 up, 7 in 2024-02-10T07:37:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:21 smithi070 ceph-mon[29548]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:21.773 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-02-10T07:37:22.366 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:22 smithi155 ceph-mon[32688]: osdmap e49: 7 total, 7 up, 7 in 2024-02-10T07:37:22.366 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:22 smithi155 ceph-mon[32688]: osdmap e50: 7 total, 7 up, 7 in 2024-02-10T07:37:22.409 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:37:22.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:22 smithi070 ceph-mon[29548]: osdmap e49: 7 total, 7 up, 7 in 2024-02-10T07:37:22.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:22 smithi070 ceph-mon[29548]: osdmap e50: 7 total, 7 up, 7 in 2024-02-10T07:37:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:22 smithi070 ceph-mon[39923]: osdmap e49: 7 total, 7 up, 7 in 2024-02-10T07:37:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:22 smithi070 ceph-mon[39923]: osdmap e50: 7 total, 7 up, 7 in 2024-02-10T07:37:23.662 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi155 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi155 to 5696M 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:23.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.664 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: osdmap e51: 7 total, 7 up, 7 in 2024-02-10T07:37:23.664 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:23 smithi155 ceph-mon[32688]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi155 2024-02-10T07:37:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi155 to 5696M 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: osdmap e51: 7 total, 7 up, 7 in 2024-02-10T07:37:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[39923]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi155 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi155 to 5696M 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:23.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:23.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:23.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: osdmap e51: 7 total, 7 up, 7 in 2024-02-10T07:37:23.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:23 smithi070 ceph-mon[29548]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:23.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.893+0000 7f6bddc65700 1 -- 172.21.15.155:0/3429960108 >> v1:172.21.15.155:6789/0 conn(0x7f6bd8109ca0 legacy=0x7f6bd810c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:23.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.893+0000 7f6bddc65700 1 -- 172.21.15.155:0/3429960108 shutdown_connections 2024-02-10T07:37:23.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.893+0000 7f6bddc65700 1 -- 172.21.15.155:0/3429960108 >> 172.21.15.155:0/3429960108 conn(0x7f6bd80fd0d0 msgr2=0x7f6bd80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:23.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.893+0000 7f6bddc65700 1 -- 172.21.15.155:0/3429960108 shutdown_connections 2024-02-10T07:37:23.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.893+0000 7f6bddc65700 1 -- 172.21.15.155:0/3429960108 wait complete. 2024-02-10T07:37:23.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bddc65700 1 Processor -- start 2024-02-10T07:37:23.895 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bddc65700 1 -- start start 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bddc65700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6bd81a4140 con 0x7f6bd8105f60 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bddc65700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6bd81a5340 con 0x7f6bd8109ca0 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bddc65700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6bd81a6540 con 0x7f6bd8101b30 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bd7fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f6bd8109ca0 0x7f6bd81a2840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:58600/0 (socket says 172.21.15.155:58600) 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bd7fff700 1 -- 172.21.15.155:0/2025026530 learned_addr learned my addr 172.21.15.155:0/2025026530 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.894+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801333919 0 0) 0x7f6bd81a5340 con 0x7f6bd8109ca0 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc00035b0 con 0x7f6bd8109ca0 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409458199 0 0) 0x7f6bd81a4140 con 0x7f6bd8105f60 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd81a5340 con 0x7f6bd8105f60 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436274168 0 0) 0x7f6bd81a6540 con 0x7f6bd8101b30 2024-02-10T07:37:23.898 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd81a4140 con 0x7f6bd8101b30 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 401723186 0 0) 0x7f6bc00035b0 con 0x7f6bd8109ca0 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6bd81a6540 con 0x7f6bd8109ca0 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6bd00017b0 con 0x7f6bd8109ca0 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 157163712 0 0) 0x7f6bd81a5340 con 0x7f6bd8105f60 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6bc00035b0 con 0x7f6bd8105f60 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6bc80017e0 con 0x7f6bd8105f60 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 635464166 0 0) 0x7f6bd81a4140 con 0x7f6bd8101b30 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6bd81a5340 con 0x7f6bd8101b30 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6bcc0017b0 con 0x7f6bd8101b30 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 973871562 0 0) 0x7f6bd81a6540 con 0x7f6bd8109ca0 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 >> v1:172.21.15.70:6790/0 conn(0x7f6bd8101b30 legacy=0x7f6bd80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 >> v1:172.21.15.70:6789/0 conn(0x7f6bd8105f60 legacy=0x7f6bd819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:23.899 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd81a7740 con 0x7f6bd8109ca0 2024-02-10T07:37:23.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd81a4330 con 0x7f6bd8109ca0 2024-02-10T07:37:23.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd81a4830 con 0x7f6bd8109ca0 2024-02-10T07:37:23.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.895+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f6bd0001ef0 con 0x7f6bd8109ca0 2024-02-10T07:37:23.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.896+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6bd00054f0 con 0x7f6bd8109ca0 2024-02-10T07:37:23.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.896+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc4005150 con 0x7f6bd8109ca0 2024-02-10T07:37:23.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.897+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f6bd0003c10 con 0x7f6bd8109ca0 2024-02-10T07:37:23.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.897+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 3789+0+0 (unknown 505829085 0 0) 0x7f6bd0088490 con 0x7f6bd8109ca0 2024-02-10T07:37:23.902 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:23.902+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6bd0052580 con 0x7f6bd8109ca0 2024-02-10T07:37:24.104 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:24.103+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bc4000bc0 con 0x7f6bc006e8b0 2024-02-10T07:37:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:37:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:37:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:24 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:24.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:37:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:37:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:24 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:37:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:37:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:24 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:25.794 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:25 smithi155 ceph-mon[32688]: from='client.24307 v1:172.21.15.155:0/2025026530' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:25.794 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:25 smithi155 ceph-mon[32688]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:25 smithi070 ceph-mon[29548]: from='client.24307 v1:172.21.15.155:0/2025026530' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:25 smithi070 ceph-mon[29548]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:25 smithi070 ceph-mon[39923]: from='client.24307 v1:172.21.15.155:0/2025026530' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:25 smithi070 ceph-mon[39923]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:26.693 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/3323482526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]: dispatch 2024-02-10T07:37:26.694 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/3323482526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]: dispatch 2024-02-10T07:37:26.694 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[29548]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]: dispatch 2024-02-10T07:37:26.694 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[29548]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]': finished 2024-02-10T07:37:26.694 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[29548]: osdmap e52: 8 total, 7 up, 8 in 2024-02-10T07:37:26.694 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[39923]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]: dispatch 2024-02-10T07:37:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[39923]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]': finished 2024-02-10T07:37:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[39923]: osdmap e52: 8 total, 7 up, 8 in 2024-02-10T07:37:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:26 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/3323482526' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]: dispatch 2024-02-10T07:37:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:26 smithi155 ceph-mon[32688]: from='client.24313 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]: dispatch 2024-02-10T07:37:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:26 smithi155 ceph-mon[32688]: from='client.24313 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2"}]': finished 2024-02-10T07:37:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:26 smithi155 ceph-mon[32688]: osdmap e52: 8 total, 7 up, 8 in 2024-02-10T07:37:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:26 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:27 smithi070 ceph-mon[39923]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-10T07:37:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/2953513853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:37:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:27 smithi070 ceph-mon[29548]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-10T07:37:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/2953513853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:37:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:27 smithi155 ceph-mon[32688]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-02-10T07:37:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/2953513853' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:37:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:29 smithi070 ceph-mon[29548]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-10T07:37:30.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:29 smithi070 ceph-mon[39923]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-10T07:37:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:29 smithi155 ceph-mon[32688]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-10T07:37:31.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:31 smithi155 ceph-mon[32688]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-10T07:37:32.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:31 smithi070 ceph-mon[29548]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-10T07:37:32.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:31 smithi070 ceph-mon[39923]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-10T07:37:34.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:33 smithi155 ceph-mon[32688]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:34.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:33 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T07:37:34.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:33 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:34.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:33 smithi155 ceph-mon[32688]: Deploying daemon osd.7 on smithi155 2024-02-10T07:37:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[29548]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T07:37:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[29548]: Deploying daemon osd.7 on smithi155 2024-02-10T07:37:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[39923]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T07:37:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:33 smithi070 ceph-mon[39923]: Deploying daemon osd.7 on smithi155 2024-02-10T07:37:36.231 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:35 smithi155 ceph-mon[32688]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:36.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:35 smithi070 ceph-mon[29548]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:36.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:35 smithi070 ceph-mon[39923]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-10T07:37:37.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:37 smithi155 ceph-mon[32688]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:37 smithi070 ceph-mon[29548]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:37 smithi070 ceph-mon[39923]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[29548]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:39 smithi070 ceph-mon[39923]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:39 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:39 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:39 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:39 smithi155 ceph-mon[32688]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: from='osd.7 v1:172.21.15.155:6812/2127871237' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T07:37:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T07:37:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: from='osd.7 v1:172.21.15.155:6812/2127871237' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:41 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: from='osd.7 v1:172.21.15.155:6812/2127871237' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:41 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:42.468 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2024-02-10T07:37:42.468 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:42.467+0000 7f6bd4ff9700 1 -- 172.21.15.155:0/2025026530 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 687725071) 0x7f6bc4000bc0 con 0x7f6bc006e8b0 2024-02-10T07:37:42.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:42.469+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 >> v1:172.21.15.70:6800/3858935040 conn(0x7f6bc006e8b0 legacy=0x7f6bc0070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:42.472 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:42.469+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 >> v1:172.21.15.155:6789/0 conn(0x7f6bd8109ca0 legacy=0x7f6bd81a2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:42.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:42.469+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 shutdown_connections 2024-02-10T07:37:42.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:42.469+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 >> 172.21.15.155:0/2025026530 conn(0x7f6bd80fd0d0 msgr2=0x7f6bd810c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:42.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:42.469+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 shutdown_connections 2024-02-10T07:37:42.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:42.469+0000 7f6bddc65700 1 -- 172.21.15.155:0/2025026530 wait complete. 2024-02-10T07:37:43.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T07:37:43.047 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: osdmap e53: 8 total, 7 up, 8 in 2024-02-10T07:37:43.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='osd.7 v1:172.21.15.155:6812/2127871237' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:43.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:43.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:43.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:43.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:43.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:43.048 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:42 smithi155 ceph-mon[32688]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:37:43.073 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.7.service 2024-02-10T07:37:43.075 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-10T07:37:43.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd stat -f json 2024-02-10T07:37:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T07:37:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: osdmap e53: 8 total, 7 up, 8 in 2024-02-10T07:37:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='osd.7 v1:172.21.15.155:6812/2127871237' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[29548]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: osdmap e53: 8 total, 7 up, 8 in 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='osd.7 v1:172.21.15.155:6812/2127871237' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:42 smithi070 ceph-mon[39923]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-10T07:37:43.328 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:37:44.070 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:43 smithi155 ceph-mon[32688]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:44.070 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:43 smithi155 ceph-mon[32688]: osdmap e54: 8 total, 7 up, 8 in 2024-02-10T07:37:44.070 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:43 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.070 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:43 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.071 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:43 smithi155 ceph-mon[32688]: osd.7 v1:172.21.15.155:6812/2127871237 boot 2024-02-10T07:37:44.071 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:43 smithi155 ceph-mon[32688]: osdmap e55: 8 total, 8 up, 8 in 2024-02-10T07:37:44.071 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:43 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[29548]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[29548]: osdmap e54: 8 total, 7 up, 8 in 2024-02-10T07:37:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[29548]: osd.7 v1:172.21.15.155:6812/2127871237 boot 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[29548]: osdmap e55: 8 total, 8 up, 8 in 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[39923]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[39923]: osdmap e54: 8 total, 7 up, 8 in 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[39923]: osd.7 v1:172.21.15.155:6812/2127871237 boot 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[39923]: osdmap e55: 8 total, 8 up, 8 in 2024-02-10T07:37:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:43 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:37:45.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:44 smithi070 ceph-mon[29548]: purged_snaps scrub starts 2024-02-10T07:37:45.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:44 smithi070 ceph-mon[29548]: purged_snaps scrub ok 2024-02-10T07:37:45.046 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:44 smithi070 ceph-mon[39923]: purged_snaps scrub starts 2024-02-10T07:37:45.046 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:44 smithi070 ceph-mon[39923]: purged_snaps scrub ok 2024-02-10T07:37:45.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.044+0000 7f67de892700 1 -- 172.21.15.70:0/2366515846 >> v1:172.21.15.70:6789/0 conn(0x7f67d8101b10 legacy=0x7f67d8101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:45.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.044+0000 7f67de892700 1 -- 172.21.15.70:0/2366515846 shutdown_connections 2024-02-10T07:37:45.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.044+0000 7f67de892700 1 -- 172.21.15.70:0/2366515846 >> 172.21.15.70:0/2366515846 conn(0x7f67d80fd0d0 msgr2=0x7f67d80ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:45.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.044+0000 7f67de892700 1 -- 172.21.15.70:0/2366515846 shutdown_connections 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.044+0000 7f67de892700 1 -- 172.21.15.70:0/2366515846 wait complete. 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.045+0000 7f67de892700 1 Processor -- start 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.045+0000 7f67de892700 1 -- start start 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.045+0000 7f67de892700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f67d81a3f50 con 0x7f67d8109c80 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.045+0000 7f67de892700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f67d81a5150 con 0x7f67d8105f40 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.045+0000 7f67de892700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f67d81a6350 con 0x7f67d8101b10 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d7fff700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f67d8101b10 0x7f67d80759e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:56194/0 (socket says 172.21.15.70:56194) 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d7fff700 1 -- 172.21.15.70:0/2066742145 learned_addr learned my addr 172.21.15.70:0/2066742145 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344803665 0 0) 0x7f67d81a3f50 con 0x7f67d8109c80 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c00035b0 con 0x7f67d8109c80 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152212788 0 0) 0x7f67d81a6350 con 0x7f67d8101b10 2024-02-10T07:37:45.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d81a3f50 con 0x7f67d8101b10 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631330140 0 0) 0x7f67d81a5150 con 0x7f67d8105f40 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d81a6350 con 0x7f67d8105f40 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3010852758 0 0) 0x7f67d81a3f50 con 0x7f67d8101b10 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67d81a5150 con 0x7f67d8101b10 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 102476102 0 0) 0x7f67c00035b0 con 0x7f67d8109c80 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67d81a3f50 con 0x7f67d8109c80 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f67cc004400 con 0x7f67d8101b10 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3680588095 0 0) 0x7f67d81a6350 con 0x7f67d8105f40 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f67c00035b0 con 0x7f67d8105f40 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f67d0001bb0 con 0x7f67d8109c80 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f67c80017f0 con 0x7f67d8105f40 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 904661556 0 0) 0x7f67d81a5150 con 0x7f67d8101b10 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 >> v1:172.21.15.155:6789/0 conn(0x7f67d8105f40 legacy=0x7f67d819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:45.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 >> v1:172.21.15.70:6789/0 conn(0x7f67d8109c80 legacy=0x7f67d81a2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d81a7550 con 0x7f67d8101b10 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d81a4140 con 0x7f67d8101b10 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d81a4640 con 0x7f67d8101b10 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f67cc003b10 con 0x7f67d8101b10 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.046+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f67cc004f20 con 0x7f67d8101b10 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.047+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c4005150 con 0x7f67d8101b10 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.048+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f67cc005230 con 0x7f67d8101b10 2024-02-10T07:37:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.048+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(56..56 src has 1..56) v4 ==== 4110+0+0 (unknown 3617630378 0 0) 0x7f67cc0882d0 con 0x7f67d8101b10 2024-02-10T07:37:45.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.055+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f67cc052280 con 0x7f67d8101b10 2024-02-10T07:37:45.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:44 smithi155 ceph-mon[32688]: purged_snaps scrub starts 2024-02-10T07:37:45.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:44 smithi155 ceph-mon[32688]: purged_snaps scrub ok 2024-02-10T07:37:45.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.248+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f67c4005400 con 0x7f67d8101b10 2024-02-10T07:37:45.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.248+0000 7f67d57fa700 1 -- 172.21.15.70:0/2066742145 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v56) v1 ==== 74+0+130 (unknown 401554801 0 1608325874) 0x7f67cc0541b0 con 0x7f67d8101b10 2024-02-10T07:37:45.250 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:37:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.250+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 >> v1:172.21.15.70:6800/3858935040 conn(0x7f67c006e9d0 legacy=0x7f67c0070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.250+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 >> v1:172.21.15.70:6790/0 conn(0x7f67d8101b10 legacy=0x7f67d80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.251+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 shutdown_connections 2024-02-10T07:37:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.251+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 >> 172.21.15.70:0/2066742145 conn(0x7f67d80fd0d0 msgr2=0x7f67d8108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.251+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 shutdown_connections 2024-02-10T07:37:45.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:45.251+0000 7f67de892700 1 -- 172.21.15.70:0/2066742145 wait complete. 2024-02-10T07:37:45.823 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1707550663,"num_in_osds":8,"osd_in_since":1707550646,"num_remapped_pgs":1} 2024-02-10T07:37:45.823 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T07:37:46.077 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:37:46.197 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:46.197 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: osdmap e56: 8 total, 8 up, 8 in 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2066742145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:46.198 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: osdmap e56: 8 total, 8 up, 8 in 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2066742145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:46.199 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:45 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: osdmap e56: 8 total, 8 up, 8 in 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2066742145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:37:46.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:46.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:46.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:45 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:47.211 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[39923]: Detected new or changed devices on smithi155 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi155 to 4272M 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[39923]: osdmap e57: 8 total, 8 up, 8 in 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[39923]: osdmap e58: 8 total, 8 up, 8 in 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[29548]: Detected new or changed devices on smithi155 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi155 to 4272M 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[29548]: osdmap e57: 8 total, 8 up, 8 in 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:47.212 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:46 smithi070 ceph-mon[29548]: osdmap e58: 8 total, 8 up, 8 in 2024-02-10T07:37:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:46 smithi155 ceph-mon[32688]: Detected new or changed devices on smithi155 2024-02-10T07:37:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:46 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi155 to 4272M 2024-02-10T07:37:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:46 smithi155 ceph-mon[32688]: osdmap e57: 8 total, 8 up, 8 in 2024-02-10T07:37:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:46 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:46 smithi155 ceph-mon[32688]: osdmap e58: 8 total, 8 up, 8 in 2024-02-10T07:37:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.709+0000 7f62764a4700 1 -- 172.21.15.70:0/2776400833 >> v1:172.21.15.70:6790/0 conn(0x7f627010a970 legacy=0x7f627010cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.709+0000 7f62764a4700 1 -- 172.21.15.70:0/2776400833 shutdown_connections 2024-02-10T07:37:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.709+0000 7f62764a4700 1 -- 172.21.15.70:0/2776400833 >> 172.21.15.70:0/2776400833 conn(0x7f62700fd5d0 msgr2=0x7f62700ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.709+0000 7f62764a4700 1 -- 172.21.15.70:0/2776400833 shutdown_connections 2024-02-10T07:37:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.709+0000 7f62764a4700 1 -- 172.21.15.70:0/2776400833 wait complete. 2024-02-10T07:37:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f62764a4700 1 Processor -- start 2024-02-10T07:37:47.710 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f62764a4700 1 -- start start 2024-02-10T07:37:47.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f62764a4700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62701a6980 con 0x7f627010e540 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f62764a4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62701a7b80 con 0x7f6270107ac0 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f62764a4700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62701a8d80 con 0x7f627010a970 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f6274a41700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f627010e540 0x7f62701a5080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57096/0 (socket says 172.21.15.70:57096) 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f6274a41700 1 -- 172.21.15.70:0/1395574674 learned_addr learned my addr 172.21.15.70:0/1395574674 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724269852 0 0) 0x7f62701a6980 con 0x7f627010e540 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62580031f0 con 0x7f627010e540 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471193134 0 0) 0x7f62701a8d80 con 0x7f627010a970 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62701a6980 con 0x7f627010a970 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.710+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166262014 0 0) 0x7f62701a7b80 con 0x7f6270107ac0 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62701a8d80 con 0x7f6270107ac0 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3299837988 0 0) 0x7f62580031f0 con 0x7f627010e540 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62701a7b80 con 0x7f627010e540 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6268001d70 con 0x7f627010e540 2024-02-10T07:37:47.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 296986911 0 0) 0x7f62701a6980 con 0x7f627010a970 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62580031f0 con 0x7f627010a970 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f62640042d0 con 0x7f627010a970 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4181445023 0 0) 0x7f62701a8d80 con 0x7f6270107ac0 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62701a6980 con 0x7f6270107ac0 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6260001830 con 0x7f6270107ac0 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 118767761 0 0) 0x7f62701a7b80 con 0x7f627010e540 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 >> v1:172.21.15.70:6790/0 conn(0x7f627010a970 legacy=0x7f62701a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 >> v1:172.21.15.155:6789/0 conn(0x7f6270107ac0 legacy=0x7f627010db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62701a9f80 con 0x7f627010e540 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62701a8f10 con 0x7f627010e540 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f62701a9390 con 0x7f627010e540 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f6268001910 con 0x7f627010e540 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.711+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6268005f70 con 0x7f627010e540 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.712+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625c005150 con 0x7f627010e540 2024-02-10T07:37:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.712+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f6268003a00 con 0x7f627010e540 2024-02-10T07:37:47.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.713+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 115522752 0 0) 0x7f6268088290 con 0x7f627010e540 2024-02-10T07:37:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.717+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f62680521f0 con 0x7f627010e540 2024-02-10T07:37:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.910+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f625c005400 con 0x7f627010e540 2024-02-10T07:37:47.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.913+0000 7f626d7fa700 1 -- 172.21.15.70:0/1395574674 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+11497 (unknown 1651271748 0 3612713405) 0x7f6268054120 con 0x7f627010e540 2024-02-10T07:37:47.913 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:37:47.913 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":59,"fsid":"70142e68-c7e6-11ee-95b8-87774f69a715","created":"2024-02-10T07:32:26.906892+0000","modified":"2024-02-10T07:37:47.691169+0000","last_up_change":"2024-02-10T07:37:43.894235+0000","last_in_change":"2024-02-10T07:37:26.615461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T07:35:41.128169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21f7c994-f25f-4bfe-ba6b-bcb06c637113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":3723357830}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":3723357830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":3723357830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":3723357830}]},"public_addr":"172.21.15.70:6801/3723357830","cluster_addr":"172.21.15.70:6802/3723357830","heartbeat_back_addr":"172.21.15.70:6804/3723357830","heartbeat_front_addr":"172.21.15.70:6803/3723357830","state":["exists","up"]},{"osd":1,"uuid":"a4164b54-8dde-4071-846d-959548c7fbe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":918939980}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":918939980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":918939980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":918939980}]},"public_addr":"172.21.15.70:6805/918939980","cluster_addr":"172.21.15.70:6806/918939980","heartbeat_back_addr":"172.21.15.70:6808/918939980","heartbeat_front_addr":"172.21.15.70:6807/918939980","state":["exists","up"]},{"osd":2,"uuid":"b5ea7978-5f0f-4ba4-a881-ed48efdd57d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":3606696222}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":3606696222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":3606696222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":3606696222}]},"public_addr":"172.21.15.70:6809/3606696222","cluster_addr":"172.21.15.70:6810/3606696222","heartbeat_back_addr":"172.21.15.70:6812/3606696222","heartbeat_front_addr":"172.21.15.70:6811/3606696222","state":["exists","up"]},{"osd":3,"uuid":"5953e938-5967-420e-9262-2c2ba2ab3bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":3512826399}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":3512826399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6816","nonce":3512826399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":3512826399}]},"public_addr":"172.21.15.70:6813/3512826399","cluster_addr":"172.21.15.70:6814/3512826399","heartbeat_back_addr":"172.21.15.70:6816/3512826399","heartbeat_front_addr":"172.21.15.70:6815/3512826399","state":["exists","up"]},{"osd":4,"uuid":"f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":1901321685}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":1901321685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":1901321685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":1901321685}]},"public_addr":"172.21.15.155:6800/1901321685","cluster_addr":"172.21.15.155:6801/1901321685","heartbeat_back_addr":"172.21.15.155:6803/1901321685","heartbeat_front_addr":"172.21.15.155:6802/1901321685","state":["exists","up"]},{"osd":5,"uuid":"4f131619-da44-4168-a436-8a7525448564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":969979851}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":969979851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":969979851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":969979851}]},"public_addr":"172.21.15.155:6804/969979851","cluster_addr":"172.21.15.155:6805/969979851","heartbeat_back_addr":"172.21.15.155:6807/969979851","heartbeat_front_addr":"172.21.15.155:6806/969979851","state":["exists","up"]},{"osd":6,"uuid":"4d2ab766-9537-43a1-8f08-621786e62715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":3655535506}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":3655535506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":3655535506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":3655535506}]},"public_addr":"172.21.15.155:6808/3655535506","cluster_addr":"172.21.15.155:6809/3655535506","heartbeat_back_addr":"172.21.15.155:6811/3655535506","heartbeat_front_addr":"172.21.15.155:6810/3655535506","state":["exists","up"]},{"osd":7,"uuid":"2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":2127871237}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":2127871237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":2127871237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":2127871237}]},"public_addr":"172.21.15.155:6812/2127871237","cluster_addr":"172.21.15.155:6813/2127871237","heartbeat_back_addr":"172.21.15.155:6815/2127871237","heartbeat_front_addr":"172.21.15.155:6814/2127871237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:34:48.078516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:35:13.167179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:35:37.892306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:03.973193+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:27.390006+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:52.444572+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:37:17.537509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:37:42.663953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6800/1990687880":"2024-02-11T07:33:26.790782+0000","172.21.15.70:6800/1444653888":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/2812082303":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/3407186454":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/3824912336":"2024-02-11T07:33:26.790782+0000","172.21.15.70:0/3785821604":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/664263038":"2024-02-11T07:33:26.790782+0000","172.21.15.70:0/3841311618":"2024-02-11T07:33:26.790782+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T07:37:47.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.915+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 >> v1:172.21.15.70:6800/3858935040 conn(0x7f625806e930 legacy=0x7f6258070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.915+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 >> v1:172.21.15.70:6789/0 conn(0x7f627010e540 legacy=0x7f62701a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.915+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 shutdown_connections 2024-02-10T07:37:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.915+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 >> 172.21.15.70:0/1395574674 conn(0x7f62700fd5d0 msgr2=0x7f62701001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.915+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 shutdown_connections 2024-02-10T07:37:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:47.915+0000 7f62764a4700 1 -- 172.21.15.70:0/1395574674 wait complete. 2024-02-10T07:37:48.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:47 smithi070 ceph-mon[29548]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-02-10T07:37:48.218 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:47 smithi070 ceph-mon[29548]: osdmap e59: 8 total, 8 up, 8 in 2024-02-10T07:37:48.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:47 smithi070 ceph-mon[39923]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-02-10T07:37:48.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:47 smithi070 ceph-mon[39923]: osdmap e59: 8 total, 8 up, 8 in 2024-02-10T07:37:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:47 smithi155 ceph-mon[32688]: pgmap v178: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-02-10T07:37:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:47 smithi155 ceph-mon[32688]: osdmap e59: 8 total, 8 up, 8 in 2024-02-10T07:37:48.499 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-10T07:35:41.128169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-10T07:37:48.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-10T07:37:48.734 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:37:49.149 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1395574674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:37:49.149 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1395574674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:37:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1395574674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:37:50.020 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:49 smithi070 ceph-mon[29548]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-10T07:37:50.021 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:49 smithi070 ceph-mon[39923]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.234+0000 7f62a7198700 1 -- 172.21.15.70:0/2645719901 >> v1:172.21.15.70:6789/0 conn(0x7f62a0107b60 legacy=0x7f62a0107f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.234+0000 7f62a7198700 1 -- 172.21.15.70:0/2645719901 shutdown_connections 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.234+0000 7f62a7198700 1 -- 172.21.15.70:0/2645719901 >> 172.21.15.70:0/2645719901 conn(0x7f62a00fd600 msgr2=0x7f62a00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a7198700 1 -- 172.21.15.70:0/2645719901 shutdown_connections 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a7198700 1 -- 172.21.15.70:0/2645719901 wait complete. 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a7198700 1 Processor -- start 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a7198700 1 -- start start 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a7198700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62a01a8bd0 con 0x7f62a0107b60 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a7198700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62a01a9d70 con 0x7f62a010a980 2024-02-10T07:37:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a7198700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62a01aaf70 con 0x7f62a010e550 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a4f34700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f62a0107b60 0x7f62a0075910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57120/0 (socket says 172.21.15.70:57120) 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a5735700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f62a010e550 0x7f62a01a7420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:43754/0 (socket says 172.21.15.70:43754) 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.235+0000 7f62a4f34700 1 -- 172.21.15.70:0/2026535572 learned_addr learned my addr 172.21.15.70:0/2026535572 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457345153 0 0) 0x7f62a01a8bd0 con 0x7f62a0107b60 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62900035b0 con 0x7f62a0107b60 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617217983 0 0) 0x7f62a01aaf70 con 0x7f62a010e550 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a01a8bd0 con 0x7f62a010e550 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090169916 0 0) 0x7f62a01a9d70 con 0x7f62a010a980 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a01aaf70 con 0x7f62a010a980 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 965735534 0 0) 0x7f62900035b0 con 0x7f62a0107b60 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62a01a9d70 con 0x7f62a0107b60 2024-02-10T07:37:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f62940015d0 con 0x7f62a0107b60 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2905568577 0 0) 0x7f62a01a8bd0 con 0x7f62a010e550 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62900035b0 con 0x7f62a010e550 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6298001b30 con 0x7f62a010e550 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2389455454 0 0) 0x7f62a01aaf70 con 0x7f62a010a980 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62a01a8bd0 con 0x7f62a010a980 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f628c001850 con 0x7f62a010a980 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1393381677 0 0) 0x7f62a01a9d70 con 0x7f62a0107b60 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 >> v1:172.21.15.70:6790/0 conn(0x7f62a010e550 legacy=0x7f62a01a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 >> v1:172.21.15.155:6789/0 conn(0x7f62a010a980 legacy=0x7f62a01a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a01ac170 con 0x7f62a0107b60 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a01a8dc0 con 0x7f62a0107b60 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.236+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f6294001080 con 0x7f62a0107b60 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.237+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6294004e30 con 0x7f62a0107b60 2024-02-10T07:37:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.237+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a01a9240 con 0x7f62a0107b60 2024-02-10T07:37:50.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.237+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a01031d0 con 0x7f62a0107b60 2024-02-10T07:37:50.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.238+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f6294003680 con 0x7f62a0107b60 2024-02-10T07:37:50.241 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.239+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 115522752 0 0) 0x7f62940880d0 con 0x7f62a0107b60 2024-02-10T07:37:50.243 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.243+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6294052090 con 0x7f62a0107b60 2024-02-10T07:37:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:49 smithi155 ceph-mon[32688]: pgmap v180: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-10T07:37:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.434+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62a00753d0 con 0x7f62a0107b60 2024-02-10T07:37:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.435+0000 7f629dffb700 1 -- 172.21.15.70:0/2026535572 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (unknown 4013917427 0 2170607528) 0x7f6294053fc0 con 0x7f62a0107b60 2024-02-10T07:37:50.435 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-02-10T07:37:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.437+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 >> v1:172.21.15.70:6800/3858935040 conn(0x7f629006e980 legacy=0x7f6290070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.437+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 >> v1:172.21.15.70:6789/0 conn(0x7f62a0107b60 legacy=0x7f62a0075910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.437+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 shutdown_connections 2024-02-10T07:37:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.437+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 >> 172.21.15.70:0/2026535572 conn(0x7f62a00fd600 msgr2=0x7f62a01111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.437+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 shutdown_connections 2024-02-10T07:37:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:37:50.437+0000 7f62a7198700 1 -- 172.21.15.70:0/2026535572 wait complete. 2024-02-10T07:37:51.130 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2026535572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T07:37:51.130 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2026535572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T07:37:51.131 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi070 2024-02-10T07:37:51.131 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi070=foo.a' 2024-02-10T07:37:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2026535572' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T07:37:51.383 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:37:52.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:51 smithi070 ceph-mon[29548]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T07:37:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:51 smithi070 ceph-mon[39923]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T07:37:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:51 smithi155 ceph-mon[32688]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-02-10T07:37:52.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.992+0000 7f5d658fc700 1 -- 172.21.15.155:0/3314938415 >> v1:172.21.15.155:6789/0 conn(0x7f5d60102600 legacy=0x7f5d601049f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:52.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.992+0000 7f5d658fc700 1 -- 172.21.15.155:0/3314938415 shutdown_connections 2024-02-10T07:37:52.993 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.992+0000 7f5d658fc700 1 -- 172.21.15.155:0/3314938415 >> 172.21.15.155:0/3314938415 conn(0x7f5d600fbfb0 msgr2=0x7f5d600fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.992+0000 7f5d658fc700 1 -- 172.21.15.155:0/3314938415 shutdown_connections 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.993+0000 7f5d658fc700 1 -- 172.21.15.155:0/3314938415 wait complete. 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.993+0000 7f5d658fc700 1 Processor -- start 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.993+0000 7f5d658fc700 1 -- start start 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.993+0000 7f5d658fc700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d60197990 con 0x7f5d60106110 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.993+0000 7f5d658fc700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d601a3130 con 0x7f5d60109c50 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.993+0000 7f5d658fc700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d601a42d0 con 0x7f5d60102600 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d5f7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5d60109c50 0x7f5d601a0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:34776/0 (socket says 172.21.15.155:34776) 2024-02-10T07:37:52.994 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d5f7fe700 1 -- 172.21.15.155:0/2362976074 learned_addr learned my addr 172.21.15.155:0/2362976074 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907926650 0 0) 0x7f5d601a3130 con 0x7f5d60109c50 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d480035b0 con 0x7f5d60109c50 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233641932 0 0) 0x7f5d601a42d0 con 0x7f5d60102600 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d601a3130 con 0x7f5d60102600 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485165658 0 0) 0x7f5d60197990 con 0x7f5d60106110 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d601a42d0 con 0x7f5d60106110 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3840035878 0 0) 0x7f5d480035b0 con 0x7f5d60109c50 2024-02-10T07:37:52.996 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d60197990 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f5d58001c70 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3444219517 0 0) 0x7f5d60197990 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 >> v1:172.21.15.70:6790/0 conn(0x7f5d60102600 legacy=0x7f5d60196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 >> v1:172.21.15.70:6789/0 conn(0x7f5d60106110 legacy=0x7f5d6019d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d601a5470 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d601a2180 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d601a2680 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f5d58003eb0 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.994+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f5d58005fd0 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.995+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c005150 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.996+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f5d58003a20 con 0x7f5d60109c50 2024-02-10T07:37:52.997 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:52.996+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 115522752 0 0) 0x7f5d58088460 con 0x7f5d60109c50 2024-02-10T07:37:53.000 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.001+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f5d580523c0 con 0x7f5d60109c50 2024-02-10T07:37:53.201 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.201+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi070=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d4c000bc0 con 0x7f5d4806e540 2024-02-10T07:37:53.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.208+0000 7f5d648fa700 1 -- 172.21.15.155:0/2362976074 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (unknown 0 0 1123153589) 0x7f5d4c000bc0 con 0x7f5d4806e540 2024-02-10T07:37:53.208 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled rgw.foo.a update... 2024-02-10T07:37:53.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.211+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 >> v1:172.21.15.70:6800/3858935040 conn(0x7f5d4806e540 legacy=0x7f5d48070a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:53.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.211+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 >> v1:172.21.15.155:6789/0 conn(0x7f5d60109c50 legacy=0x7f5d601a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:53.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.211+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 shutdown_connections 2024-02-10T07:37:53.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.211+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 >> 172.21.15.155:0/2362976074 conn(0x7f5d600fbfb0 msgr2=0x7f5d60104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:53.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.211+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 shutdown_connections 2024-02-10T07:37:53.211 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:53.211+0000 7f5d658fc700 1 -- 172.21.15.155:0/2362976074 wait complete. 2024-02-10T07:37:53.896 DEBUG:teuthology.orchestra.run.smithi070:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@rgw.foo.a.service 2024-02-10T07:37:53.898 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi155 2024-02-10T07:37:53.898 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-02-10T07:37:54.140 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T07:37:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:53 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:53 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:55.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:54 smithi155 ceph-mon[32688]: from='client.24352 v1:172.21.15.155:0/2362976074' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi070=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:55.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:54 smithi155 ceph-mon[32688]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-02-10T07:37:55.140 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:54 smithi155 ceph-mon[32688]: Deploying daemon rgw.foo.a on smithi070 2024-02-10T07:37:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:54 smithi070 ceph-mon[29548]: from='client.24352 v1:172.21.15.155:0/2362976074' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi070=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:54 smithi070 ceph-mon[29548]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-02-10T07:37:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:54 smithi070 ceph-mon[29548]: Deploying daemon rgw.foo.a on smithi070 2024-02-10T07:37:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:54 smithi070 ceph-mon[39923]: from='client.24352 v1:172.21.15.155:0/2362976074' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi070=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:37:55.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:54 smithi070 ceph-mon[39923]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-02-10T07:37:55.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:54 smithi070 ceph-mon[39923]: Deploying daemon rgw.foo.a on smithi070 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.628+0000 7fa9770ce700 1 -- 172.21.15.155:0/3046608359 >> v1:172.21.15.155:6789/0 conn(0x7fa970109c60 legacy=0x7fa97010c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.628+0000 7fa9770ce700 1 -- 172.21.15.155:0/3046608359 shutdown_connections 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.628+0000 7fa9770ce700 1 -- 172.21.15.155:0/3046608359 >> 172.21.15.155:0/3046608359 conn(0x7fa9700fb750 msgr2=0x7fa9700fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.628+0000 7fa9770ce700 1 -- 172.21.15.155:0/3046608359 shutdown_connections 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.628+0000 7fa9770ce700 1 -- 172.21.15.155:0/3046608359 wait complete. 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.629+0000 7fa9770ce700 1 Processor -- start 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.629+0000 7fa9770ce700 1 -- start start 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.629+0000 7fa9770ce700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa970110db0 con 0x7fa970106000 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.629+0000 7fa9770ce700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa970197ac0 con 0x7fa9700ffdf0 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.629+0000 7fa9770ce700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa9701a4260 con 0x7fa970109c60 2024-02-10T07:37:55.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.629+0000 7fa974e6a700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa9700ffdf0 0x7fa970196f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:34792/0 (socket says 172.21.15.155:34792) 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.629+0000 7fa974e6a700 1 -- 172.21.15.155:0/1319824779 learned_addr learned my addr 172.21.15.155:0/1319824779 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931689970 0 0) 0x7fa970197ac0 con 0x7fa9700ffdf0 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9600037f0 con 0x7fa9700ffdf0 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804589220 0 0) 0x7fa970110db0 con 0x7fa970106000 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa970197ac0 con 0x7fa970106000 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628476242 0 0) 0x7fa9701a4260 con 0x7fa970109c60 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa970110db0 con 0x7fa970109c60 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 60425656 0 0) 0x7fa9600037f0 con 0x7fa9700ffdf0 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9701a4260 con 0x7fa9700ffdf0 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fa964001910 con 0x7fa9700ffdf0 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 703035644 0 0) 0x7fa970197ac0 con 0x7fa970106000 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9600037f0 con 0x7fa970106000 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fa95c0018b0 con 0x7fa970106000 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 11153949 0 0) 0x7fa9701a4260 con 0x7fa9700ffdf0 2024-02-10T07:37:55.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 >> v1:172.21.15.70:6790/0 conn(0x7fa970109c60 legacy=0x7fa9701a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 >> v1:172.21.15.70:6789/0 conn(0x7fa970106000 legacy=0x7fa97019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9701a5400 con 0x7fa9700ffdf0 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9701a4450 con 0x7fa9700ffdf0 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fa964001ca0 con 0x7fa9700ffdf0 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fa964005c70 con 0x7fa9700ffdf0 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.630+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9701a4950 con 0x7fa9700ffdf0 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.631+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa970068610 con 0x7fa9700ffdf0 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.632+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7fa964003950 con 0x7fa9700ffdf0 2024-02-10T07:37:55.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.632+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 115522752 0 0) 0x7fa964087e40 con 0x7fa9700ffdf0 2024-02-10T07:37:55.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.637+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fa964051ee0 con 0x7fa9700ffdf0 2024-02-10T07:37:55.830 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:55.830+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fa97010ed30 con 0x7fa9700ffdf0 2024-02-10T07:37:56.087 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 07:37:56 smithi070 systemd[1]: Starting Ceph rgw.foo.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:37:56.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:56.138+0000 7fa96dffb700 1 -- 172.21.15.155:0/1319824779 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v60) v1 ==== 160+0+0 (unknown 701681216 0 0) 0x7fa964053e10 con 0x7fa9700ffdf0 2024-02-10T07:37:56.140 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:56.141+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 >> v1:172.21.15.70:6800/3858935040 conn(0x7fa96006e860 legacy=0x7fa960070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:56.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:56.141+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 >> v1:172.21.15.155:6789/0 conn(0x7fa9700ffdf0 legacy=0x7fa970196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:37:56.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:56.141+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 shutdown_connections 2024-02-10T07:37:56.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:56.141+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 >> 172.21.15.155:0/1319824779 conn(0x7fa9700fb750 msgr2=0x7fa97010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:37:56.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:56.141+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 shutdown_connections 2024-02-10T07:37:56.141 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:37:56.141+0000 7fa9770ce700 1 -- 172.21.15.155:0/1319824779 wait complete. 2024-02-10T07:37:56.141 INFO:teuthology.orchestra.run.smithi155.stderr:pool 'datapool' created 2024-02-10T07:37:56.365 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:56 smithi070 ceph-mon[39923]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:56.366 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:56 smithi070 ceph-mon[29548]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:56.366 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 07:37:56 smithi070 podman[66900]: 2024-02-10 07:37:56.129748612 +0000 UTC m=+0.017721398 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:37:56.366 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 07:37:56 smithi070 podman[66900]: 2024-02-10 07:37:56.244816746 +0000 UTC m=+0.132789513 container create 9000c620813b7a71066a449ef7f67c6beec7e0546a0843965fa1becf3cda5cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-10T07:37:56.427 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:56 smithi155 ceph-mon[32688]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:56.741 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 07:37:56 smithi070 podman[66900]: 2024-02-10 07:37:56.473447793 +0000 UTC m=+0.361420574 container init 9000c620813b7a71066a449ef7f67c6beec7e0546a0843965fa1becf3cda5cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-290212a) 2024-02-10T07:37:56.741 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 07:37:56 smithi070 podman[66900]: 2024-02-10 07:37:56.476039299 +0000 UTC m=+0.364012067 container start 9000c620813b7a71066a449ef7f67c6beec7e0546a0843965fa1becf3cda5cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.build-date=20240131, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:37:56.741 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 07:37:56 smithi070 bash[66900]: 9000c620813b7a71066a449ef7f67c6beec7e0546a0843965fa1becf3cda5cb7 2024-02-10T07:37:56.741 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 07:37:56 smithi070 systemd[1]: Started Ceph rgw.foo.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:37:56.745 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- rbd pool init datapool 2024-02-10T07:37:56.985 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:37:57.347 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/1319824779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T07:37:57.347 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='client.24358 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T07:37:57.347 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='client.24358 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: osdmap e60: 8 total, 8 up, 8 in 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: pgmap v185: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:57.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:57 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/1319824779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T07:37:57.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='client.24358 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T07:37:57.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='client.24358 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T07:37:57.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: osdmap e60: 8 total, 8 up, 8 in 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: pgmap v185: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/1319824779' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T07:37:57.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='client.24358 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='client.24358 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: osdmap e60: 8 total, 8 up, 8 in 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: pgmap v185: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:37:57.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:57 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:58.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:58 smithi155 ceph-mon[32688]: osdmap e61: 8 total, 8 up, 8 in 2024-02-10T07:37:58.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1801764334' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T07:37:58.370 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:58 smithi155 ceph-mon[32688]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:37:58.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:58 smithi070 ceph-mon[29548]: osdmap e61: 8 total, 8 up, 8 in 2024-02-10T07:37:58.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1801764334' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T07:37:58.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:58 smithi070 ceph-mon[29548]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:37:58.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:58 smithi070 ceph-mon[39923]: osdmap e61: 8 total, 8 up, 8 in 2024-02-10T07:37:58.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1801764334' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T07:37:58.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:58 smithi070 ceph-mon[39923]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:37:59.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1801764334' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T07:37:59.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: osdmap e62: 8 total, 8 up, 8 in 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: Checking dashboard <-> RGW credentials 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/2620082552' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: from='client.24382 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[29548]: pgmap v188: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1801764334' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: osdmap e62: 8 total, 8 up, 8 in 2024-02-10T07:37:59.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:59.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:59.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:59.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:59.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: Checking dashboard <-> RGW credentials 2024-02-10T07:37:59.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/2620082552' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T07:37:59.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: from='client.24382 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T07:37:59.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:37:59 smithi070 ceph-mon[39923]: pgmap v188: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:37:59.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1801764334' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T07:37:59.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: osdmap e62: 8 total, 8 up, 8 in 2024-02-10T07:37:59.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:59.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:37:59.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:37:59.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:37:59.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: Checking dashboard <-> RGW credentials 2024-02-10T07:37:59.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/2620082552' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T07:37:59.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: from='client.24382 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-10T07:37:59.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:37:59 smithi155 ceph-mon[32688]: pgmap v188: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:00.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[29548]: from='client.24382 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[29548]: osdmap e63: 8 total, 8 up, 8 in 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[39923]: from='client.24382 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[39923]: osdmap e63: 8 total, 8 up, 8 in 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:00 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:00 smithi155 ceph-mon[32688]: from='client.24382 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-10T07:38:00.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:00 smithi155 ceph-mon[32688]: osdmap e63: 8 total, 8 up, 8 in 2024-02-10T07:38:00.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:00.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:00 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:38:01.415 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:38:01.416 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:01 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:38:01.416 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:01 smithi155 ceph-mon[32688]: osdmap e64: 8 total, 8 up, 8 in 2024-02-10T07:38:01.416 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:01 smithi155 ceph-mon[32688]: pgmap v191: 68 pgs: 2 creating+activating, 9 creating+peering, 34 unknown, 23 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:38:01.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:38:01.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:38:01.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[29548]: osdmap e64: 8 total, 8 up, 8 in 2024-02-10T07:38:01.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[29548]: pgmap v191: 68 pgs: 2 creating+activating, 9 creating+peering, 34 unknown, 23 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:38:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:38:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:38:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[39923]: osdmap e64: 8 total, 8 up, 8 in 2024-02-10T07:38:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:01 smithi070 ceph-mon[39923]: pgmap v191: 68 pgs: 2 creating+activating, 9 creating+peering, 34 unknown, 23 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:38:01.711 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.155 --placement '1;smithi155=iscsi.a' 2024-02-10T07:38:01.955 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:38:02.410 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:02 smithi155 ceph-mon[32688]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T07:38:02.411 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:02 smithi155 ceph-mon[32688]: Cluster is now healthy 2024-02-10T07:38:02.411 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:02 smithi155 ceph-mon[32688]: osdmap e65: 8 total, 8 up, 8 in 2024-02-10T07:38:02.411 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.411 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:02 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.411 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[29548]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T07:38:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[29548]: Cluster is now healthy 2024-02-10T07:38:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[29548]: osdmap e65: 8 total, 8 up, 8 in 2024-02-10T07:38:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[39923]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-10T07:38:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[39923]: Cluster is now healthy 2024-02-10T07:38:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[39923]: osdmap e65: 8 total, 8 up, 8 in 2024-02-10T07:38:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:38:03.433 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:03 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:38:03.433 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:38:03.433 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:03 smithi155 ceph-mon[32688]: osdmap e66: 8 total, 8 up, 8 in 2024-02-10T07:38:03.434 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:03 smithi155 ceph-mon[32688]: pgmap v194: 100 pgs: 2 creating+activating, 13 creating+peering, 30 unknown, 55 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:38:03.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.459+0000 7fe68a451700 1 -- 172.21.15.155:0/220261879 >> v1:172.21.15.155:6789/0 conn(0x7fe684101b20 legacy=0x7fe684101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:03.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.459+0000 7fe68a451700 1 -- 172.21.15.155:0/220261879 shutdown_connections 2024-02-10T07:38:03.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.459+0000 7fe68a451700 1 -- 172.21.15.155:0/220261879 >> 172.21.15.155:0/220261879 conn(0x7fe6840fd0c0 msgr2=0x7fe6840ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:03.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.459+0000 7fe68a451700 1 -- 172.21.15.155:0/220261879 shutdown_connections 2024-02-10T07:38:03.459 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.459+0000 7fe68a451700 1 -- 172.21.15.155:0/220261879 wait complete. 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe68a451700 1 Processor -- start 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe68a451700 1 -- start start 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe68a451700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe684110da0 con 0x7fe684101b20 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe68a451700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe6841a51f0 con 0x7fe684105f50 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe68a451700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe6841a63f0 con 0x7fe684109c90 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe67bfff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe684105f50 0x7fe68419ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43828/0 (socket says 172.21.15.155:43828) 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe67bfff700 1 -- 172.21.15.155:0/3158097859 learned_addr learned my addr 172.21.15.155:0/3158097859 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332965761 0 0) 0x7fe6841a51f0 con 0x7fe684105f50 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6700035b0 con 0x7fe684105f50 2024-02-10T07:38:03.462 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804081260 0 0) 0x7fe684110da0 con 0x7fe684101b20 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6841a51f0 con 0x7fe684101b20 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624510008 0 0) 0x7fe6841a63f0 con 0x7fe684109c90 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.460+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe684110da0 con 0x7fe684109c90 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 45296924 0 0) 0x7fe6700035b0 con 0x7fe684105f50 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6841a63f0 con 0x7fe684105f50 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fe66c001930 con 0x7fe684105f50 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3879053388 0 0) 0x7fe6841a51f0 con 0x7fe684101b20 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6700035b0 con 0x7fe684101b20 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 445970919 0 0) 0x7fe684110da0 con 0x7fe684109c90 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6841a51f0 con 0x7fe684109c90 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fe674001430 con 0x7fe684101b20 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fe67c001a90 con 0x7fe684109c90 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3115733044 0 0) 0x7fe6841a63f0 con 0x7fe684105f50 2024-02-10T07:38:03.463 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 >> v1:172.21.15.70:6790/0 conn(0x7fe684109c90 legacy=0x7fe6841a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 >> v1:172.21.15.70:6789/0 conn(0x7fe684101b20 legacy=0x7fe6840759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6841a75f0 con 0x7fe684105f50 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6841a41e0 con 0x7fe684105f50 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6841a46e0 con 0x7fe684105f50 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fe66c003b30 con 0x7fe684105f50 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.461+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fe66c005c20 con 0x7fe684105f50 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.462+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe664005120 con 0x7fe684105f50 2024-02-10T07:38:03.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.462+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7fe66c0036a0 con 0x7fe684105f50 2024-02-10T07:38:03.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.463+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(67..67 src has 1..67) v4 ==== 5915+0+0 (unknown 1473908613 0 0) 0x7fe66c0886d0 con 0x7fe684105f50 2024-02-10T07:38:03.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.467+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fe66c051f70 con 0x7fe684105f50 2024-02-10T07:38:03.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:38:03.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:38:03.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[29548]: osdmap e66: 8 total, 8 up, 8 in 2024-02-10T07:38:03.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[29548]: pgmap v194: 100 pgs: 2 creating+activating, 13 creating+peering, 30 unknown, 55 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:38:03.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:38:03.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:38:03.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[39923]: osdmap e66: 8 total, 8 up, 8 in 2024-02-10T07:38:03.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:03 smithi070 ceph-mon[39923]: pgmap v194: 100 pgs: 2 creating+activating, 13 creating+peering, 30 unknown, 55 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:38:03.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.673+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fe664000c60 con 0x7fe67006e9f0 2024-02-10T07:38:03.683 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.682+0000 7fe681ffb700 1 -- 172.21.15.155:0/3158097859 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (unknown 0 0 803663096) 0x7fe664000c60 con 0x7fe67006e9f0 2024-02-10T07:38:03.683 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled iscsi.datapool update... 2024-02-10T07:38:03.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.684+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 >> v1:172.21.15.70:6800/3858935040 conn(0x7fe67006e9f0 legacy=0x7fe670070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:03.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.684+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 >> v1:172.21.15.155:6789/0 conn(0x7fe684105f50 legacy=0x7fe68419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:03.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.685+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 shutdown_connections 2024-02-10T07:38:03.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.685+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 >> 172.21.15.155:0/3158097859 conn(0x7fe6840fd0c0 msgr2=0x7fe684108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:03.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.685+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 shutdown_connections 2024-02-10T07:38:03.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:03.685+0000 7fe68a451700 1 -- 172.21.15.155:0/3158097859 wait complete. 2024-02-10T07:38:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:04 smithi155 ceph-mon[32688]: osdmap e67: 8 total, 8 up, 8 in 2024-02-10T07:38:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:04 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:04 smithi155 ceph-mon[32688]: from='client.24418 v1:172.21.15.155:0/3158097859' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:04 smithi155 ceph-mon[32688]: Saving service iscsi.datapool spec with placement smithi155=iscsi.a;count:1 2024-02-10T07:38:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:04 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:04.306 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-02-10T07:38:04.306 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:38:04.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-10T07:38:04.341 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:38:04.341 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-10T07:38:04.370 DEBUG:teuthology.orchestra.run.smithi155:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@iscsi.iscsi.a.service 2024-02-10T07:38:04.413 INFO:tasks.cephadm:Adding prometheus.a on smithi155 2024-02-10T07:38:04.414 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply prometheus '1;smithi155=a' 2024-02-10T07:38:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[29548]: osdmap e67: 8 total, 8 up, 8 in 2024-02-10T07:38:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[29548]: from='client.24418 v1:172.21.15.155:0/3158097859' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[29548]: Saving service iscsi.datapool spec with placement smithi155=iscsi.a;count:1 2024-02-10T07:38:04.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:04.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[39923]: osdmap e67: 8 total, 8 up, 8 in 2024-02-10T07:38:04.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:38:04.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[39923]: from='client.24418 v1:172.21.15.155:0/3158097859' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:04.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[39923]: Saving service iscsi.datapool spec with placement smithi155=iscsi.a;count:1 2024-02-10T07:38:04.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:04 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:04.691 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:38:05.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:38:05.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:38:05.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: osdmap e68: 8 total, 8 up, 8 in 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: pgmap v197: 132 pgs: 3 creating+activating, 14 creating+peering, 32 unknown, 83 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[39923]: osdmap e69: 8 total, 8 up, 8 in 2024-02-10T07:38:05.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: osdmap e68: 8 total, 8 up, 8 in 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: pgmap v197: 132 pgs: 3 creating+activating, 14 creating+peering, 32 unknown, 83 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:38:05.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:05 smithi070 ceph-mon[29548]: osdmap e69: 8 total, 8 up, 8 in 2024-02-10T07:38:05.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:38:05.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:38:05.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: osdmap e68: 8 total, 8 up, 8 in 2024-02-10T07:38:05.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/261812307' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:38:05.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: pgmap v197: 132 pgs: 3 creating+activating, 14 creating+peering, 32 unknown, 83 active+clean; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:38:05.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: from='client.34409 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:38:05.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/375850049' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:38:05.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:05 smithi155 ceph-mon[32688]: osdmap e69: 8 total, 8 up, 8 in 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.202+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/1649382598 >> v1:172.21.15.155:6789/0 conn(0x7f2ce0108ba0 legacy=0x7f2ce010b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.202+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/1649382598 shutdown_connections 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.202+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/1649382598 >> 172.21.15.155:0/1649382598 conn(0x7f2ce00fbff0 msgr2=0x7f2ce00fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.202+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/1649382598 shutdown_connections 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.202+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/1649382598 wait complete. 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.202+0000 7f2ce5a7c700 1 Processor -- start 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.202+0000 7f2ce5a7c700 1 -- start start 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2ce5a7c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2ce01a4210 con 0x7f2ce0104e60 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2ce5a7c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2ce01a5410 con 0x7f2ce0108ba0 2024-02-10T07:38:06.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2ce5a7c700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2ce01a6610 con 0x7f2ce0100a30 2024-02-10T07:38:06.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdffff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f2ce0108ba0 0x7f2ce01a2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:43854/0 (socket says 172.21.15.155:43854) 2024-02-10T07:38:06.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdffff700 1 -- 172.21.15.155:0/3926926720 learned_addr learned my addr 172.21.15.155:0/3926926720 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:38:06.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622517576 0 0) 0x7f2ce01a5410 con 0x7f2ce0108ba0 2024-02-10T07:38:06.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ccc0035b0 con 0x7f2ce0108ba0 2024-02-10T07:38:06.205 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402054477 0 0) 0x7f2ce01a4210 con 0x7f2ce0104e60 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce01a5410 con 0x7f2ce0104e60 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905545219 0 0) 0x7f2ce01a6610 con 0x7f2ce0100a30 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce01a4210 con 0x7f2ce0100a30 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 502982678 0 0) 0x7f2ccc0035b0 con 0x7f2ce0108ba0 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.203+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ce01a6610 con 0x7f2ce0108ba0 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f2cd8001c40 con 0x7f2ce0108ba0 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3184334825 0 0) 0x7f2ce01a4210 con 0x7f2ce0100a30 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ccc0035b0 con 0x7f2ce0100a30 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2703141713 0 0) 0x7f2ce01a5410 con 0x7f2ce0104e60 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ce01a4210 con 0x7f2ce0104e60 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f2cd0001830 con 0x7f2ce0100a30 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f2cc80017e0 con 0x7f2ce0104e60 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 491215721 0 0) 0x7f2ce01a6610 con 0x7f2ce0108ba0 2024-02-10T07:38:06.206 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 >> v1:172.21.15.70:6790/0 conn(0x7f2ce0100a30 legacy=0x7f2ce0106d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 >> v1:172.21.15.70:6789/0 conn(0x7f2ce0104e60 legacy=0x7f2ce019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce01a7810 con 0x7f2ce0108ba0 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce01a4400 con 0x7f2ce0108ba0 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce01a4900 con 0x7f2ce0108ba0 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f2cd8001540 con 0x7f2ce0108ba0 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.204+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f2cd80052b0 con 0x7f2ce0108ba0 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.205+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc0005150 con 0x7f2ce0108ba0 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.205+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f2cd80054b0 con 0x7f2ce0108ba0 2024-02-10T07:38:06.207 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.206+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(69..69 src has 1..69) v4 ==== 5942+0+0 (unknown 1778814318 0 0) 0x7f2cd8088c10 con 0x7f2ce0108ba0 2024-02-10T07:38:06.210 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.210+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f2cd8052570 con 0x7f2ce0108ba0 2024-02-10T07:38:06.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.428+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc0000bc0 con 0x7f2ccc06ead0 2024-02-10T07:38:06.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.436+0000 7f2cdd7fa700 1 -- 172.21.15.155:0/3926926720 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (unknown 0 0 1342662408) 0x7f2cc0000bc0 con 0x7f2ccc06ead0 2024-02-10T07:38:06.436 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled prometheus update... 2024-02-10T07:38:06.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.438+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 >> v1:172.21.15.70:6800/3858935040 conn(0x7f2ccc06ead0 legacy=0x7f2ccc070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:06.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.438+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 >> v1:172.21.15.155:6789/0 conn(0x7f2ce0108ba0 legacy=0x7f2ce01a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:06.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.438+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 shutdown_connections 2024-02-10T07:38:06.438 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.438+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 >> 172.21.15.155:0/3926926720 conn(0x7f2ce00fbff0 msgr2=0x7f2ce006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:06.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.438+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 shutdown_connections 2024-02-10T07:38:06.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:06.438+0000 7f2ce5a7c700 1 -- 172.21.15.155:0/3926926720 wait complete. 2024-02-10T07:38:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T07:38:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T07:38:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[29548]: Deploying daemon iscsi.iscsi.a on smithi155 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:38:06.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:06 smithi070 ceph-mon[39923]: Deploying daemon iscsi.iscsi.a on smithi155 2024-02-10T07:38:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T07:38:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T07:38:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:06 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:38:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:06 smithi155 ceph-mon[32688]: Deploying daemon iscsi.iscsi.a on smithi155 2024-02-10T07:38:07.422 DEBUG:teuthology.orchestra.run.smithi155:prometheus.a> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@prometheus.a.service 2024-02-10T07:38:07.424 INFO:tasks.cephadm:Adding node-exporter.a on smithi070 2024-02-10T07:38:07.425 INFO:tasks.cephadm:Adding node-exporter.b on smithi155 2024-02-10T07:38:07.425 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply node-exporter '2;smithi070=a;smithi155=b' 2024-02-10T07:38:07.694 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:07 smithi155 ceph-mon[32688]: from='client.24439 v1:172.21.15.155:0/3926926720' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:07.694 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:07 smithi155 ceph-mon[32688]: Saving service prometheus spec with placement smithi155=a;count:1 2024-02-10T07:38:07.694 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:07 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:07.695 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:07 smithi155 ceph-mon[32688]: pgmap v199: 132 pgs: 1 creating+activating, 14 creating+peering, 7 unknown, 110 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 4.3 KiB/s wr, 96 op/s 2024-02-10T07:38:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[29548]: from='client.24439 v1:172.21.15.155:0/3926926720' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[29548]: Saving service prometheus spec with placement smithi155=a;count:1 2024-02-10T07:38:07.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:07.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[29548]: pgmap v199: 132 pgs: 1 creating+activating, 14 creating+peering, 7 unknown, 110 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 4.3 KiB/s wr, 96 op/s 2024-02-10T07:38:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[39923]: from='client.24439 v1:172.21.15.155:0/3926926720' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[39923]: Saving service prometheus spec with placement smithi155=a;count:1 2024-02-10T07:38:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:07 smithi070 ceph-mon[39923]: pgmap v199: 132 pgs: 1 creating+activating, 14 creating+peering, 7 unknown, 110 active+clean; 580 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 4.3 KiB/s wr, 96 op/s 2024-02-10T07:38:08.040 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:38:09.705 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.704+0000 7f7ed28ed700 1 -- 172.21.15.155:0/476296834 >> v1:172.21.15.155:6789/0 conn(0x7f7ecc104e40 legacy=0x7f7ecc107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:09.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.704+0000 7f7ed28ed700 1 -- 172.21.15.155:0/476296834 shutdown_connections 2024-02-10T07:38:09.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.704+0000 7f7ed28ed700 1 -- 172.21.15.155:0/476296834 >> 172.21.15.155:0/476296834 conn(0x7f7ecc0fbfb0 msgr2=0x7f7ecc0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:09.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.704+0000 7f7ed28ed700 1 -- 172.21.15.155:0/476296834 shutdown_connections 2024-02-10T07:38:09.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.704+0000 7f7ed28ed700 1 -- 172.21.15.155:0/476296834 wait complete. 2024-02-10T07:38:09.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ed28ed700 1 Processor -- start 2024-02-10T07:38:09.706 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ed28ed700 1 -- start start 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ed28ed700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ecc10d8b0 con 0x7f7ecc108b80 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ed28ed700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ecc1a95f0 con 0x7f7ecc100a10 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ed28ed700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ecc1aa7f0 con 0x7f7ecc104e40 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ecbfff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f7ecc100a10 0x7f7ecc106d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44322/0 (socket says 172.21.15.155:44322) 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ecbfff700 1 -- 172.21.15.155:0/1915931379 learned_addr learned my addr 172.21.15.155:0/1915931379 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293410897 0 0) 0x7f7ecc1a95f0 con 0x7f7ecc100a10 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb40037f0 con 0x7f7ecc100a10 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.705+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023290475 0 0) 0x7f7ecc1aa7f0 con 0x7f7ecc104e40 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ecc1a95f0 con 0x7f7ecc104e40 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854784971 0 0) 0x7f7ecc10d8b0 con 0x7f7ecc108b80 2024-02-10T07:38:09.708 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ecc1aa7f0 con 0x7f7ecc108b80 2024-02-10T07:38:09.709 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1489969109 0 0) 0x7f7eb40037f0 con 0x7f7ecc100a10 2024-02-10T07:38:09.709 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ecc10d8b0 con 0x7f7ecc100a10 2024-02-10T07:38:09.709 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7ebc001970 con 0x7f7ecc100a10 2024-02-10T07:38:09.709 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 473738554 0 0) 0x7f7ecc10d8b0 con 0x7f7ecc100a10 2024-02-10T07:38:09.709 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 >> v1:172.21.15.70:6790/0 conn(0x7f7ecc104e40 legacy=0x7f7ecc1a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:09.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 >> v1:172.21.15.70:6789/0 conn(0x7f7ecc108b80 legacy=0x7f7ecc1a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:09.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc1ab9f0 con 0x7f7ecc100a10 2024-02-10T07:38:09.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc1a8600 con 0x7f7ecc100a10 2024-02-10T07:38:09.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc1a8b00 con 0x7f7ecc100a10 2024-02-10T07:38:09.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f7ebc001c80 con 0x7f7ecc100a10 2024-02-10T07:38:09.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.706+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7ebc005ce0 con 0x7f7ecc100a10 2024-02-10T07:38:09.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.707+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb8005150 con 0x7f7ecc100a10 2024-02-10T07:38:09.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.707+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f7ebc003950 con 0x7f7ecc100a10 2024-02-10T07:38:09.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.708+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(69..69 src has 1..69) v4 ==== 5942+0+0 (unknown 1778814318 0 0) 0x7f7ebc088820 con 0x7f7ecc100a10 2024-02-10T07:38:09.713 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.712+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f7ebc051e70 con 0x7f7ecc100a10 2024-02-10T07:38:09.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.924+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi070=a;smithi155=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eb8000bc0 con 0x7f7eb406e4c0 2024-02-10T07:38:09.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.931+0000 7f7ec97fa700 1 -- 172.21.15.155:0/1915931379 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (unknown 0 0 240551134) 0x7f7eb8000bc0 con 0x7f7eb406e4c0 2024-02-10T07:38:09.932 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled node-exporter update... 2024-02-10T07:38:09.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.933+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 >> v1:172.21.15.70:6800/3858935040 conn(0x7f7eb406e4c0 legacy=0x7f7eb4070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:09.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.933+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 >> v1:172.21.15.155:6789/0 conn(0x7f7ecc100a10 legacy=0x7f7ecc106d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:09.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.934+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 shutdown_connections 2024-02-10T07:38:09.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.934+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 >> 172.21.15.155:0/1915931379 conn(0x7f7ecc0fbfb0 msgr2=0x7f7ecc10c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:09.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.934+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 shutdown_connections 2024-02-10T07:38:09.935 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:09.934+0000 7f7ed28ed700 1 -- 172.21.15.155:0/1915931379 wait complete. 2024-02-10T07:38:10.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:09 smithi155 ceph-mon[32688]: pgmap v200: 132 pgs: 1 creating+activating, 10 creating+peering, 121 active+clean; 581 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 4.5 KiB/s wr, 111 op/s 2024-02-10T07:38:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:09 smithi070 ceph-mon[29548]: pgmap v200: 132 pgs: 1 creating+activating, 10 creating+peering, 121 active+clean; 581 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 4.5 KiB/s wr, 111 op/s 2024-02-10T07:38:10.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:09 smithi070 ceph-mon[39923]: pgmap v200: 132 pgs: 1 creating+activating, 10 creating+peering, 121 active+clean; 581 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 4.5 KiB/s wr, 111 op/s 2024-02-10T07:38:10.544 DEBUG:teuthology.orchestra.run.smithi070:node-exporter.a> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.a.service 2024-02-10T07:38:10.547 DEBUG:teuthology.orchestra.run.smithi155:node-exporter.b> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.b.service 2024-02-10T07:38:10.550 INFO:tasks.cephadm:Adding alertmanager.a on smithi070 2024-02-10T07:38:10.550 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply alertmanager '1;smithi070=a' 2024-02-10T07:38:11.178 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: from='client.24445 v1:172.21.15.155:0/1915931379' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi070=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:11.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: Saving service node-exporter spec with placement smithi070=a;smithi155=b;count:2 2024-02-10T07:38:11.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.179 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:10 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: from='client.24445 v1:172.21.15.155:0/1915931379' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi070=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: Saving service node-exporter spec with placement smithi070=a;smithi155=b;count:2 2024-02-10T07:38:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: from='client.24445 v1:172.21.15.155:0/1915931379' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi070=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: Saving service node-exporter spec with placement smithi070=a;smithi155=b;count:2 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:10 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:11.302 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:38:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:11 smithi155 ceph-mon[32688]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-10T07:38:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:11 smithi155 ceph-mon[32688]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s rd, 5.0 KiB/s wr, 147 op/s 2024-02-10T07:38:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/2865741554' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T07:38:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:11 smithi155 ceph-mon[32688]: osdmap e70: 8 total, 8 up, 8 in 2024-02-10T07:38:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:11 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[29548]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-10T07:38:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[29548]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s rd, 5.0 KiB/s wr, 147 op/s 2024-02-10T07:38:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/2865741554' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T07:38:12.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[29548]: osdmap e70: 8 total, 8 up, 8 in 2024-02-10T07:38:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[39923]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-10T07:38:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[39923]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 61 KiB/s rd, 5.0 KiB/s wr, 147 op/s 2024-02-10T07:38:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/2865741554' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T07:38:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[39923]: osdmap e70: 8 total, 8 up, 8 in 2024-02-10T07:38:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:11 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:12.885 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.884+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3967625548 >> v1:172.21.15.155:6789/0 conn(0x7f1ebc100a10 legacy=0x7f1ebc100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.884+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3967625548 shutdown_connections 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.884+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3967625548 >> 172.21.15.155:0/3967625548 conn(0x7f1ebc0fbfb0 msgr2=0x7f1ebc0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.884+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3967625548 shutdown_connections 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.884+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3967625548 wait complete. 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.884+0000 7f1ec2fe1700 1 Processor -- start 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1ec2fe1700 1 -- start start 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1ec2fe1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1ebc06a5c0 con 0x7f1ebc100a10 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1ec2fe1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1ebc1a31b0 con 0x7f1ebc108b80 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1ec2fe1700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1ebc1a43b0 con 0x7f1ebc104e40 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1ec157e700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1ebc108b80 0x7f1ebc1a0a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:44420/0 (socket says 172.21.15.155:44420) 2024-02-10T07:38:12.886 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1ec157e700 1 -- 172.21.15.155:0/3223249901 learned_addr learned my addr 172.21.15.155:0/3223249901 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:38:12.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173109854 0 0) 0x7f1ebc1a31b0 con 0x7f1ebc108b80 2024-02-10T07:38:12.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eac0035b0 con 0x7f1ebc108b80 2024-02-10T07:38:12.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084243245 0 0) 0x7f1ebc06a5c0 con 0x7f1ebc100a10 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ebc1a31b0 con 0x7f1ebc100a10 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620783135 0 0) 0x7f1ebc1a43b0 con 0x7f1ebc104e40 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ebc06a5c0 con 0x7f1ebc104e40 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4146978736 0 0) 0x7f1eac0035b0 con 0x7f1ebc108b80 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ebc1a43b0 con 0x7f1ebc108b80 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1eb4001c70 con 0x7f1ebc108b80 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1497042178 0 0) 0x7f1ebc06a5c0 con 0x7f1ebc104e40 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1eac0035b0 con 0x7f1ebc104e40 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.885+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1ea8001720 con 0x7f1ebc104e40 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3280463945 0 0) 0x7f1ebc1a43b0 con 0x7f1ebc108b80 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 >> v1:172.21.15.70:6790/0 conn(0x7f1ebc104e40 legacy=0x7f1ebc19d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 >> v1:172.21.15.70:6789/0 conn(0x7f1ebc100a10 legacy=0x7f1ebc199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc1a55b0 con 0x7f1ebc108b80 2024-02-10T07:38:12.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc1a2200 con 0x7f1ebc108b80 2024-02-10T07:38:12.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f1eb4002000 con 0x7f1ebc108b80 2024-02-10T07:38:12.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1eb4006040 con 0x7f1ebc108b80 2024-02-10T07:38:12.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.886+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc1a2700 con 0x7f1ebc108b80 2024-02-10T07:38:12.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.887+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc068610 con 0x7f1ebc108b80 2024-02-10T07:38:12.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.887+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 91371+0+0 (unknown 2603656752 0 0) 0x7f1eb4004350 con 0x7f1ebc108b80 2024-02-10T07:38:12.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.888+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5942+0+0 (unknown 843060872 0 0) 0x7f1eb4088c70 con 0x7f1ebc108b80 2024-02-10T07:38:12.893 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.892+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f1eb40524f0 con 0x7f1ebc108b80 2024-02-10T07:38:12.942 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:12.941+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mgrmap(e 16) v1 ==== 91422+0+0 (unknown 2981307970 0 0) 0x7f1eb4051fa0 con 0x7f1ebc108b80 2024-02-10T07:38:13.107 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.106+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi070=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ebc1a2a20 con 0x7f1eac06e6e0 2024-02-10T07:38:13.114 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.113+0000 7f1eb9ffb700 1 -- 172.21.15.155:0/3223249901 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (unknown 0 0 1850065467) 0x7f1ebc1a2a20 con 0x7f1eac06e6e0 2024-02-10T07:38:13.114 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled alertmanager update... 2024-02-10T07:38:13.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.116+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 >> v1:172.21.15.70:6800/3858935040 conn(0x7f1eac06e6e0 legacy=0x7f1eac070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:13.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.116+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 >> v1:172.21.15.155:6789/0 conn(0x7f1ebc108b80 legacy=0x7f1ebc1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:13.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.116+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 shutdown_connections 2024-02-10T07:38:13.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.116+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 >> 172.21.15.155:0/3223249901 conn(0x7f1ebc0fbfb0 msgr2=0x7f1ebc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:13.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.116+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 shutdown_connections 2024-02-10T07:38:13.117 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:13.116+0000 7f1ec2fe1700 1 -- 172.21.15.155:0/3223249901 wait complete. 2024-02-10T07:38:14.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:13 smithi155 ceph-mon[32688]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T07:38:14.154 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:13 smithi155 ceph-mon[32688]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-10T07:38:14.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:13 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:14.182 DEBUG:teuthology.orchestra.run.smithi070:alertmanager.a> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@alertmanager.a.service 2024-02-10T07:38:14.185 INFO:tasks.cephadm:Adding grafana.a on smithi155 2024-02-10T07:38:14.185 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph orch apply grafana '1;smithi155=a' 2024-02-10T07:38:14.210 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:13 smithi070 ceph-mon[39923]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T07:38:14.211 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:13 smithi070 ceph-mon[39923]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-10T07:38:14.211 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:13 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:14.211 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:13 smithi070 ceph-mon[29548]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T07:38:14.211 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:13 smithi070 ceph-mon[29548]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-10T07:38:14.211 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:13 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:14.741 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:38:15.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:14 smithi155 ceph-mon[32688]: from='client.24481 v1:172.21.15.155:0/3223249901' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi070=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:15.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:14 smithi155 ceph-mon[32688]: Saving service alertmanager spec with placement smithi070=a;count:1 2024-02-10T07:38:15.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:14 smithi070 ceph-mon[39923]: from='client.24481 v1:172.21.15.155:0/3223249901' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi070=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:15.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:14 smithi070 ceph-mon[39923]: Saving service alertmanager spec with placement smithi070=a;count:1 2024-02-10T07:38:15.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:14 smithi070 ceph-mon[29548]: from='client.24481 v1:172.21.15.155:0/3223249901' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi070=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:15.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:14 smithi070 ceph-mon[29548]: Saving service alertmanager spec with placement smithi070=a;count:1 2024-02-10T07:38:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[29548]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.8 KiB/s wr, 117 op/s 2024-02-10T07:38:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[39923]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.8 KiB/s wr, 117 op/s 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:38:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:15 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:15 smithi155 ceph-mon[32688]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.8 KiB/s wr, 117 op/s 2024-02-10T07:38:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:15 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:15 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:15 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:38:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:15 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:38:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:15 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:16.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- 172.21.15.155:0/3929515281 >> v1:172.21.15.155:6789/0 conn(0x7f6fa4108bc0 legacy=0x7f6fa410b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:16.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- 172.21.15.155:0/3929515281 shutdown_connections 2024-02-10T07:38:16.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- 172.21.15.155:0/3929515281 >> 172.21.15.155:0/3929515281 conn(0x7f6fa40fc010 msgr2=0x7f6fa40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- 172.21.15.155:0/3929515281 shutdown_connections 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- 172.21.15.155:0/3929515281 wait complete. 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 Processor -- start 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- start start 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6fa406ad50 con 0x7f6fa4104e80 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6fa406af40 con 0x7f6fa4100a50 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.629+0000 7f6fab3ba700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6fa41a6670 con 0x7f6fa4108bc0 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6fa9156700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f6fa4100a50 0x7f6fa419c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35514/0 (socket says 172.21.15.155:35514) 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6fa9156700 1 -- 172.21.15.155:0/1939280093 learned_addr learned my addr 172.21.15.155:0/1939280093 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910799675 0 0) 0x7f6fa406af40 con 0x7f6fa4100a50 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f900037f0 con 0x7f6fa4100a50 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028671734 0 0) 0x7f6fa406ad50 con 0x7f6fa4104e80 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa406af40 con 0x7f6fa4104e80 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577363448 0 0) 0x7f6fa41a6670 con 0x7f6fa4108bc0 2024-02-10T07:38:16.634 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa406ad50 con 0x7f6fa4108bc0 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2349988775 0 0) 0x7f6f900037f0 con 0x7f6fa4100a50 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fa41a6670 con 0x7f6fa4100a50 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6fa00017f0 con 0x7f6fa4100a50 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1364926822 0 0) 0x7f6fa406af40 con 0x7f6fa4104e80 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6f900037f0 con 0x7f6fa4104e80 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6f940018b0 con 0x7f6fa4104e80 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 569807426 0 0) 0x7f6fa41a6670 con 0x7f6fa4100a50 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 >> v1:172.21.15.70:6790/0 conn(0x7f6fa4108bc0 legacy=0x7f6fa41a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 >> v1:172.21.15.70:6789/0 conn(0x7f6fa4104e80 legacy=0x7f6fa419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa41a7810 con 0x7f6fa4100a50 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa41a6860 con 0x7f6fa4100a50 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa41a6d60 con 0x7f6fa4100a50 2024-02-10T07:38:16.635 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f6fa0001d60 con 0x7f6fa4100a50 2024-02-10T07:38:16.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.630+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6fa0005ac0 con 0x7f6fa4100a50 2024-02-10T07:38:16.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.631+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f88005150 con 0x7f6fa4100a50 2024-02-10T07:38:16.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.632+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 16) v1 ==== 91422+0+0 (unknown 2981307970 0 0) 0x7f6fa00036a0 con 0x7f6fa4100a50 2024-02-10T07:38:16.636 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.632+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5942+0+0 (unknown 843060872 0 0) 0x7f6fa00887d0 con 0x7f6fa4100a50 2024-02-10T07:38:16.637 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.637+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6fa0051f50 con 0x7f6fa4100a50 2024-02-10T07:38:16.853 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.852+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 --> v1:172.21.15.70:6800/3858935040 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f88000bc0 con 0x7f6f9006eb40 2024-02-10T07:38:16.860 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.860+0000 7f6f9a7fc700 1 -- 172.21.15.155:0/1939280093 <== mgr.14152 v1:172.21.15.70:6800/3858935040 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (unknown 0 0 664801700) 0x7f6f88000bc0 con 0x7f6f9006eb40 2024-02-10T07:38:16.861 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled grafana update... 2024-02-10T07:38:16.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.862+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 >> v1:172.21.15.70:6800/3858935040 conn(0x7f6f9006eb40 legacy=0x7f6f90071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:16.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.862+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 >> v1:172.21.15.155:6789/0 conn(0x7f6fa4100a50 legacy=0x7f6fa419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:16.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.862+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 shutdown_connections 2024-02-10T07:38:16.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.862+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 >> 172.21.15.155:0/1939280093 conn(0x7f6fa40fc010 msgr2=0x7f6fa410b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:16.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.862+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 shutdown_connections 2024-02-10T07:38:16.863 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:16.862+0000 7f6fab3ba700 1 -- 172.21.15.155:0/1939280093 wait complete. 2024-02-10T07:38:17.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:16 smithi070 ceph-mon[29548]: Deploying daemon prometheus.a on smithi155 2024-02-10T07:38:17.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:17.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:16 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:17.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:16 smithi070 ceph-mon[39923]: Deploying daemon prometheus.a on smithi155 2024-02-10T07:38:17.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:17.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:16 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:16 smithi155 ceph-mon[32688]: Deploying daemon prometheus.a on smithi155 2024-02-10T07:38:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:16 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:17.820 DEBUG:teuthology.orchestra.run.smithi155:grafana.a> sudo journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@grafana.a.service 2024-02-10T07:38:17.822 INFO:tasks.cephadm:Setting up client nodes... 2024-02-10T07:38:17.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T07:38:18.071 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:18.193 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:17 smithi070 ceph-mon[29548]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 1.9 KiB/s wr, 69 op/s 2024-02-10T07:38:18.193 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:17 smithi070 ceph-mon[29548]: from='client.24487 v1:172.21.15.155:0/1939280093' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:18.194 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:17 smithi070 ceph-mon[29548]: Saving service grafana spec with placement smithi155=a;count:1 2024-02-10T07:38:18.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:17 smithi070 ceph-mon[39923]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 1.9 KiB/s wr, 69 op/s 2024-02-10T07:38:18.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:17 smithi070 ceph-mon[39923]: from='client.24487 v1:172.21.15.155:0/1939280093' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:18.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:17 smithi070 ceph-mon[39923]: Saving service grafana spec with placement smithi155=a;count:1 2024-02-10T07:38:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:17 smithi155 ceph-mon[32688]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 1.9 KiB/s wr, 69 op/s 2024-02-10T07:38:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:17 smithi155 ceph-mon[32688]: from='client.24487 v1:172.21.15.155:0/1939280093' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:38:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:17 smithi155 ceph-mon[32688]: Saving service grafana spec with placement smithi155=a;count:1 2024-02-10T07:38:19.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.725+0000 7fc5d131d700 1 -- 172.21.15.70:0/4131679912 >> v1:172.21.15.70:6790/0 conn(0x7fc5cc109c60 legacy=0x7fc5cc10c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:19.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.725+0000 7fc5d131d700 1 -- 172.21.15.70:0/4131679912 shutdown_connections 2024-02-10T07:38:19.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.725+0000 7fc5d131d700 1 -- 172.21.15.70:0/4131679912 >> 172.21.15.70:0/4131679912 conn(0x7fc5cc0fd0d0 msgr2=0x7fc5cc0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:19.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.725+0000 7fc5d131d700 1 -- 172.21.15.70:0/4131679912 shutdown_connections 2024-02-10T07:38:19.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.725+0000 7fc5d131d700 1 -- 172.21.15.70:0/4131679912 wait complete. 2024-02-10T07:38:19.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.725+0000 7fc5d131d700 1 Processor -- start 2024-02-10T07:38:19.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5d131d700 1 -- start start 2024-02-10T07:38:19.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5d131d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc5cc1a3f80 con 0x7fc5cc101af0 2024-02-10T07:38:19.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5d131d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc5cc1a5180 con 0x7fc5cc109c60 2024-02-10T07:38:19.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5d131d700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc5cc1a6380 con 0x7fc5cc105f20 2024-02-10T07:38:19.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5caffd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5cc101af0 0x7fc5cc0759e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36042/0 (socket says 172.21.15.70:36042) 2024-02-10T07:38:19.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5caffd700 1 -- 172.21.15.70:0/3869439090 learned_addr learned my addr 172.21.15.70:0/3869439090 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:19.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644978093 0 0) 0x7fc5cc1a3f80 con 0x7fc5cc101af0 2024-02-10T07:38:19.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b8005070 con 0x7fc5cc101af0 2024-02-10T07:38:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692420990 0 0) 0x7fc5cc1a6380 con 0x7fc5cc105f20 2024-02-10T07:38:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5cc1a3f80 con 0x7fc5cc105f20 2024-02-10T07:38:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241472444 0 0) 0x7fc5cc1a5180 con 0x7fc5cc109c60 2024-02-10T07:38:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5cc1a6380 con 0x7fc5cc109c60 2024-02-10T07:38:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 210350444 0 0) 0x7fc5b8005070 con 0x7fc5cc101af0 2024-02-10T07:38:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5cc1a5180 con 0x7fc5cc101af0 2024-02-10T07:38:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.726+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc5bc001c50 con 0x7fc5cc101af0 2024-02-10T07:38:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 61356365 0 0) 0x7fc5cc1a6380 con 0x7fc5cc109c60 2024-02-10T07:38:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5b8005070 con 0x7fc5cc109c60 2024-02-10T07:38:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc5c4003680 con 0x7fc5cc109c60 2024-02-10T07:38:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3675837289 0 0) 0x7fc5cc1a3f80 con 0x7fc5cc105f20 2024-02-10T07:38:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5cc1a6380 con 0x7fc5cc105f20 2024-02-10T07:38:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc5b40017b0 con 0x7fc5cc105f20 2024-02-10T07:38:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2908332115 0 0) 0x7fc5cc1a5180 con 0x7fc5cc101af0 2024-02-10T07:38:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 >> v1:172.21.15.70:6790/0 conn(0x7fc5cc105f20 legacy=0x7fc5cc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 >> v1:172.21.15.155:6789/0 conn(0x7fc5cc109c60 legacy=0x7fc5cc1a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc1a7580 con 0x7fc5cc101af0 2024-02-10T07:38:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc1a6570 con 0x7fc5cc101af0 2024-02-10T07:38:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc1a6a70 con 0x7fc5cc101af0 2024-02-10T07:38:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fc5bc003e90 con 0x7fc5cc101af0 2024-02-10T07:38:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.727+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fc5bc005ff0 con 0x7fc5cc101af0 2024-02-10T07:38:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.728+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac005150 con 0x7fc5cc101af0 2024-02-10T07:38:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.728+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 16) v1 ==== 91422+0+0 (unknown 2981307970 0 0) 0x7fc5bc006220 con 0x7fc5cc101af0 2024-02-10T07:38:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.729+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5942+0+0 (unknown 843060872 0 0) 0x7fc5bc088ac0 con 0x7fc5cc101af0 2024-02-10T07:38:19.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:19.733+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fc5bc052260 con 0x7fc5cc101af0 2024-02-10T07:38:20.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.007+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc5ac005400 con 0x7fc5cc101af0 2024-02-10T07:38:20.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.009+0000 7fc5c3fff700 1 -- 172.21.15.70:0/3869439090 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (unknown 57899324 0 3945499695) 0x7fc5bc054190 con 0x7fc5cc101af0 2024-02-10T07:38:20.010 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-02-10T07:38:20.011 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDsJ8dl9AOaABAAvBFnQJlv+3sioZR9d+H/ig== 2024-02-10T07:38:20.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.011+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 >> v1:172.21.15.70:6800/3858935040 conn(0x7fc5b806eb40 legacy=0x7fc5b8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:20.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.011+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 >> v1:172.21.15.70:6789/0 conn(0x7fc5cc101af0 legacy=0x7fc5cc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:20.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.011+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 shutdown_connections 2024-02-10T07:38:20.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.011+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 >> 172.21.15.70:0/3869439090 conn(0x7fc5cc0fd0d0 msgr2=0x7fc5cc108aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:20.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.012+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 shutdown_connections 2024-02-10T07:38:20.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:20.012+0000 7fc5d131d700 1 -- 172.21.15.70:0/3869439090 wait complete. 2024-02-10T07:38:20.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:19 smithi070 ceph-mon[29548]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.2 KiB/s wr, 47 op/s 2024-02-10T07:38:20.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:19 smithi070 ceph-mon[39923]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.2 KiB/s wr, 47 op/s 2024-02-10T07:38:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:19 smithi155 ceph-mon[32688]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 1.2 KiB/s wr, 47 op/s 2024-02-10T07:38:20.709 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:38:20.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-10T07:38:20.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-10T07:38:20.752 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T07:38:21.020 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:38:21.223 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3869439090' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:21.223 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3869439090' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:38:21.224 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3869439090' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:21.224 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3869439090' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:38:21.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3869439090' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:21.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3869439090' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:38:22.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:22 smithi070 ceph-mon[29548]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T07:38:22.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:22 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:22.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:22 smithi070 ceph-mon[39923]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T07:38:22.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:22 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:22.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:22 smithi155 ceph-mon[32688]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T07:38:22.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:22 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:24.380 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:24 smithi155 ceph-mon[32688]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 738 B/s rd, 92 B/s wr, 1 op/s 2024-02-10T07:38:24.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:24 smithi070 ceph-mon[29548]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 738 B/s rd, 92 B/s wr, 1 op/s 2024-02-10T07:38:24.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:24 smithi070 ceph-mon[39923]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 738 B/s rd, 92 B/s wr, 1 op/s 2024-02-10T07:38:24.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.832+0000 7f0e60fba700 1 -- 172.21.15.155:0/1611991062 >> v1:172.21.15.155:6789/0 conn(0x7f0e5c105f30 legacy=0x7f0e5c108380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:24.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.832+0000 7f0e60fba700 1 -- 172.21.15.155:0/1611991062 shutdown_connections 2024-02-10T07:38:24.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.832+0000 7f0e60fba700 1 -- 172.21.15.155:0/1611991062 >> 172.21.15.155:0/1611991062 conn(0x7f0e5c0fd0c0 msgr2=0x7f0e5c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:24.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.832+0000 7f0e60fba700 1 -- 172.21.15.155:0/1611991062 shutdown_connections 2024-02-10T07:38:24.877 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.832+0000 7f0e60fba700 1 -- 172.21.15.155:0/1611991062 wait complete. 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.832+0000 7f0e60fba700 1 Processor -- start 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.832+0000 7f0e60fba700 1 -- start start 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e60fba700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0e5c197a40 con 0x7f0e5c101b00 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e60fba700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0e5c1a3230 con 0x7f0e5c109c70 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e60fba700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0e5c1a43d0 con 0x7f0e5c105f30 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e5ad9d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0e5c109c70 0x7f0e5c1a0b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35536/0 (socket says 172.21.15.155:35536) 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e5ad9d700 1 -- 172.21.15.155:0/807684652 learned_addr learned my addr 172.21.15.155:0/807684652 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244109922 0 0) 0x7f0e5c1a3230 con 0x7f0e5c109c70 2024-02-10T07:38:24.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e440035b0 con 0x7f0e5c109c70 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222519324 0 0) 0x7f0e5c1a43d0 con 0x7f0e5c105f30 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c1a3230 con 0x7f0e5c105f30 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799178455 0 0) 0x7f0e5c197a40 con 0x7f0e5c101b00 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c1a43d0 con 0x7f0e5c101b00 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1832216475 0 0) 0x7f0e440035b0 con 0x7f0e5c109c70 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e5c197a40 con 0x7f0e5c109c70 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0e54001c10 con 0x7f0e5c109c70 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 372278206 0 0) 0x7f0e5c197a40 con 0x7f0e5c109c70 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 >> v1:172.21.15.70:6790/0 conn(0x7f0e5c105f30 legacy=0x7f0e5c19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 >> v1:172.21.15.70:6789/0 conn(0x7f0e5c101b00 legacy=0x7f0e5c196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c1a5570 con 0x7f0e5c109c70 2024-02-10T07:38:24.879 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c1a2280 con 0x7f0e5c109c70 2024-02-10T07:38:24.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.833+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c1a2780 con 0x7f0e5c109c70 2024-02-10T07:38:24.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.834+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f0e54003eb0 con 0x7f0e5c109c70 2024-02-10T07:38:24.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.834+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0e54005f40 con 0x7f0e5c109c70 2024-02-10T07:38:24.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.834+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c005150 con 0x7f0e5c109c70 2024-02-10T07:38:24.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.835+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 16) v1 ==== 91422+0+0 (unknown 2981307970 0 0) 0x7f0e54003a20 con 0x7f0e5c109c70 2024-02-10T07:38:24.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.835+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5942+0+0 (unknown 843060872 0 0) 0x7f0e54088ca0 con 0x7f0e5c109c70 2024-02-10T07:38:24.880 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:24.840+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0e54052440 con 0x7f0e5c109c70 2024-02-10T07:38:25.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.122+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0e3c005400 con 0x7f0e5c109c70 2024-02-10T07:38:25.128 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.127+0000 7f0e4b7fe700 1 -- 172.21.15.155:0/807684652 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (unknown 3508017507 0 3474680775) 0x7f0e54054370 con 0x7f0e5c109c70 2024-02-10T07:38:25.128 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2024-02-10T07:38:25.128 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDxJ8dlApB4BxAA+uOv+rDXbq4x5cxFUtXE5A== 2024-02-10T07:38:25.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.129+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 >> v1:172.21.15.70:6800/3858935040 conn(0x7f0e4406e3b0 legacy=0x7f0e44070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:25.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.129+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 >> v1:172.21.15.155:6789/0 conn(0x7f0e5c109c70 legacy=0x7f0e5c1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:25.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.129+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 shutdown_connections 2024-02-10T07:38:25.130 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.129+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 >> 172.21.15.155:0/807684652 conn(0x7f0e5c0fd0c0 msgr2=0x7f0e5c108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:25.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.129+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 shutdown_connections 2024-02-10T07:38:25.131 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-10T07:38:25.129+0000 7f0e60fba700 1 -- 172.21.15.155:0/807684652 wait complete. 2024-02-10T07:38:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:26 smithi155 ceph-mon[32688]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:38:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.155:0/807684652' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:26 smithi155 ceph-mon[32688]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:26.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:26 smithi155 ceph-mon[32688]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:38:26.351 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-10T07:38:26.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-10T07:38:26.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-10T07:38:26.398 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-10T07:38:26.398 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-10T07:38:26.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph mgr dump --format=json 2024-02-10T07:38:26.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[29548]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:38:26.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.155:0/807684652' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:26.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[29548]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:26.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[29548]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:38:26.429 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[39923]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:38:26.429 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.155:0/807684652' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:26.429 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[39923]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:38:26.429 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:26 smithi070 ceph-mon[39923]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:38:26.648 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:28.133 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.133 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.133 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:27 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.133 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:27 smithi155 ceph-mon[32688]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[39923]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T07:38:28.163 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:27 smithi070 ceph-mon[29548]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.514+0000 7f8f8317c700 1 -- 172.21.15.70:0/3240010082 >> v1:172.21.15.70:6790/0 conn(0x7f8f7c109870 legacy=0x7f8f7c10bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.514+0000 7f8f8317c700 1 -- 172.21.15.70:0/3240010082 shutdown_connections 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.514+0000 7f8f8317c700 1 -- 172.21.15.70:0/3240010082 >> 172.21.15.70:0/3240010082 conn(0x7f8f7c0fc4d0 msgr2=0x7f8f7c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.514+0000 7f8f8317c700 1 -- 172.21.15.70:0/3240010082 shutdown_connections 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.514+0000 7f8f8317c700 1 -- 172.21.15.70:0/3240010082 wait complete. 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f8317c700 1 Processor -- start 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f8317c700 1 -- start start 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f8317c700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8f7c1a8b10 con 0x7f8f7c109870 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f8317c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8f7c1a9cf0 con 0x7f8f7c10d440 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f8317c700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8f7c1aaed0 con 0x7f8f7c1069c0 2024-02-10T07:38:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f7bfff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8f7c109870 0x7f8f7c1a3bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41934/0 (socket says 172.21.15.70:41934) 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f7bfff700 1 -- 172.21.15.70:0/2939288373 learned_addr learned my addr 172.21.15.70:0/2939288373 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.515+0000 7f8f80f18700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f8f7c1069c0 0x7f8f7c10c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:44322/0 (socket says 172.21.15.70:44322) 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762366349 0 0) 0x7f8f7c1a8b10 con 0x7f8f7c109870 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f6c0031f0 con 0x7f8f7c109870 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083630858 0 0) 0x7f8f7c1aaed0 con 0x7f8f7c1069c0 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c1a8b10 con 0x7f8f7c1069c0 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666678669 0 0) 0x7f8f7c1a9cf0 con 0x7f8f7c10d440 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c1aaed0 con 0x7f8f7c10d440 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1091384904 0 0) 0x7f8f6c0031f0 con 0x7f8f7c109870 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f7c1a9cf0 con 0x7f8f7c109870 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1830922626 0 0) 0x7f8f7c1a8b10 con 0x7f8f7c1069c0 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f6c0031f0 con 0x7f8f7c1069c0 2024-02-10T07:38:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8f68001470 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8f70001850 con 0x7f8f7c1069c0 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2305115998 0 0) 0x7f8f7c1aaed0 con 0x7f8f7c10d440 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f7c1a8b10 con 0x7f8f7c10d440 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8f74001b30 con 0x7f8f7c10d440 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1062709534 0 0) 0x7f8f7c1a9cf0 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 >> v1:172.21.15.70:6790/0 conn(0x7f8f7c1069c0 legacy=0x7f8f7c10c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 >> v1:172.21.15.155:6789/0 conn(0x7f8f7c10d440 legacy=0x7f8f7c1a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c1ac0d0 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c1ab0c0 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f8f68003b10 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8f68004f30 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.516+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c1ab540 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.517+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c102100 con 0x7f8f7c109870 2024-02-10T07:38:28.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.518+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 16) v1 ==== 91422+0+0 (unknown 2981307970 0 0) 0x7f8f68003680 con 0x7f8f7c109870 2024-02-10T07:38:28.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.518+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(71..71 src has 1..71) v4 ==== 6001+0+0 (unknown 453350963 0 0) 0x7f8f68088c70 con 0x7f8f7c109870 2024-02-10T07:38:28.527 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.523+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8f680523b0 con 0x7f8f7c109870 2024-02-10T07:38:28.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.778+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8f7c109f60 con 0x7f8f7c109870 2024-02-10T07:38:28.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.782+0000 7f8f79ffb700 1 -- 172.21.15.70:0/2939288373 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+174800 (unknown 3479712753 0 487281735) 0x7f8f680542e0 con 0x7f8f7c109870 2024-02-10T07:38:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:38:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.785+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 >> v1:172.21.15.70:6800/3858935040 conn(0x7f8f6c06ea90 legacy=0x7f8f6c070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.785+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 >> v1:172.21.15.70:6789/0 conn(0x7f8f7c109870 legacy=0x7f8f7c1a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.785+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 shutdown_connections 2024-02-10T07:38:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.785+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 >> 172.21.15.70:0/2939288373 conn(0x7f8f7c0fc4d0 msgr2=0x7f8f7c06a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.785+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 shutdown_connections 2024-02-10T07:38:28.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:28.785+0000 7f8f8317c700 1 -- 172.21.15.70:0/2939288373 wait complete. 2024-02-10T07:38:28.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-10T07:38:28.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-10T07:38:28.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:28 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-02-10T07:38:28.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:28 smithi155 ceph-mon[32688]: osdmap e71: 8 total, 8 up, 8 in 2024-02-10T07:38:29.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-10T07:38:29.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-10T07:38:29.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-02-10T07:38:29.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[39923]: osdmap e71: 8 total, 8 up, 8 in 2024-02-10T07:38:29.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-10T07:38:29.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-10T07:38:29.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-02-10T07:38:29.086 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:28 smithi070 ceph-mon[29548]: osdmap e71: 8 total, 8 up, 8 in 2024-02-10T07:38:29.505 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:38:29 smithi155 systemd[1]: Started Ceph prometheus.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:38:29.519 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":3858935040}]},"active_addr":"172.21.15.70:6800/3858935040","active_change":"2024-02-10T07:33:26.790931+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.70:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1845435083}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1480684513}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2176342414}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1039112337}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1072121108}]}]} 2024-02-10T07:38:29.523 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-10T07:38:29.523 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-10T07:38:29.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T07:38:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:30.030 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[29548]: osdmap e72: 8 total, 8 up, 8 in 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939288373' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[29548]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[39923]: osdmap e72: 8 total, 8 up, 8 in 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939288373' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[39923]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.031 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.032 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:29 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T07:38:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:29 smithi155 ceph-mon[32688]: osdmap e72: 8 total, 8 up, 8 in 2024-02-10T07:38:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939288373' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T07:38:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:29 smithi155 ceph-mon[32688]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:38:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' 2024-02-10T07:38:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:29 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T07:38:30.734 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:30 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ignoring --setuser ceph since I am not root 2024-02-10T07:38:30.734 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:30 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ignoring --setgroup ceph since I am not root 2024-02-10T07:38:30.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: ignoring --setuser ceph since I am not root 2024-02-10T07:38:30.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: ignoring --setgroup ceph since I am not root 2024-02-10T07:38:31.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:30.988+0000 7fa3e1e16200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:38:31.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:31.082+0000 7fa3e1e16200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:38:31.265 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:31.008+0000 7fcc9dbff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:38:31.265 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:31.107+0000 7fcc9dbff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.508+0000 7f63bb5f9700 1 -- 172.21.15.70:0/417855838 >> v1:172.21.15.70:6789/0 conn(0x7f63b410ab20 legacy=0x7f63b410cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.508+0000 7f63bb5f9700 1 -- 172.21.15.70:0/417855838 shutdown_connections 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.508+0000 7f63bb5f9700 1 -- 172.21.15.70:0/417855838 >> 172.21.15.70:0/417855838 conn(0x7f63b40fc4c0 msgr2=0x7f63b40fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.508+0000 7f63bb5f9700 1 -- 172.21.15.70:0/417855838 shutdown_connections 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.508+0000 7f63bb5f9700 1 -- 172.21.15.70:0/417855838 wait complete. 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63bb5f9700 1 Processor -- start 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63bb5f9700 1 -- start start 2024-02-10T07:38:31.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63bb5f9700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63b41a6860 con 0x7f63b410ab20 2024-02-10T07:38:31.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63bb5f9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63b41a7a40 con 0x7f63b4102d10 2024-02-10T07:38:31.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63bb5f9700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63b41a8c20 con 0x7f63b410e630 2024-02-10T07:38:31.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63b8b94700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63b410ab20 0x7f63b41a18b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41966/0 (socket says 172.21.15.70:41966) 2024-02-10T07:38:31.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63b8b94700 1 -- 172.21.15.70:0/2893648946 learned_addr learned my addr 172.21.15.70:0/2893648946 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:31.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.509+0000 7f63b9b96700 1 --1- 172.21.15.70:0/2893648946 >> v1:172.21.15.70:6790/0 conn(0x7f63b410e630 0x7f63b41a4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:44350/0 (socket says 172.21.15.70:44350) 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377379092 0 0) 0x7f63b41a6860 con 0x7f63b410ab20 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a00049e0 con 0x7f63b410ab20 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076640797 0 0) 0x7f63b41a7a40 con 0x7f63b4102d10 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b41a6860 con 0x7f63b4102d10 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068014924 0 0) 0x7f63b41a8c20 con 0x7f63b410e630 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b41a7a40 con 0x7f63b410e630 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3476851462 0 0) 0x7f63a00049e0 con 0x7f63b410ab20 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63b41a8c20 con 0x7f63b410ab20 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f63ac001af0 con 0x7f63b410ab20 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3097423671 0 0) 0x7f63b41a6860 con 0x7f63b4102d10 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63a00049e0 con 0x7f63b4102d10 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f63b0001850 con 0x7f63b4102d10 2024-02-10T07:38:31.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1411332022 0 0) 0x7f63b41a7a40 con 0x7f63b410e630 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63b41a6860 con 0x7f63b410e630 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f63a40017f0 con 0x7f63b410e630 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2123363128 0 0) 0x7f63b41a8c20 con 0x7f63b410ab20 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 >> v1:172.21.15.70:6790/0 conn(0x7f63b410e630 legacy=0x7f63b41a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 >> v1:172.21.15.155:6789/0 conn(0x7f63b4102d10 legacy=0x7f63b410a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b41a9e20 con 0x7f63b410ab20 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b41a7c30 con 0x7f63b410ab20 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b41a80b0 con 0x7f63b410ab20 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f63ac003e90 con 0x7f63b410ab20 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.510+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f63ac005200 con 0x7f63b410ab20 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.511+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398005150 con 0x7f63b410ab20 2024-02-10T07:38:31.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.512+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 17) v1 ==== 91436+0+0 (unknown 2333988677 0 0) 0x7f63ac003a00 con 0x7f63b410ab20 2024-02-10T07:38:31.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.513+0000 7f63b9395700 1 -- 172.21.15.70:0/2893648946 >> v1:172.21.15.70:6800/3858935040 conn(0x7f63a006ead0 legacy=0x7f63a0070f90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/3858935040 2024-02-10T07:38:31.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.513+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 2587513750 0 0) 0x7f63ac088f50 con 0x7f63b410ab20 2024-02-10T07:38:31.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.522+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f63ac0525d0 con 0x7f63b410ab20 2024-02-10T07:38:31.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.714+0000 7f63b9395700 1 -- 172.21.15.70:0/2893648946 >> v1:172.21.15.70:6800/3858935040 conn(0x7f63a006ead0 legacy=0x7f63a0070f90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/3858935040 2024-02-10T07:38:31.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-mon[29548]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T07:38:31.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-mon[29548]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-10T07:38:31.715 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-mon[39923]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T07:38:31.715 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-mon[39923]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-10T07:38:31.721 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:31 smithi155 ceph-mon[32688]: from='mgr.14152 v1:172.21.15.70:0/319653729' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T07:38:31.721 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:31 smithi155 ceph-mon[32688]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-10T07:38:31.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.724+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6398005400 con 0x7f63b410ab20 2024-02-10T07:38:31.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.725+0000 7f63aa7fc700 1 -- 172.21.15.70:0/2893648946 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+20671 (unknown 1686360691 0 834537700) 0x7f63ac054500 con 0x7f63b410ab20 2024-02-10T07:38:31.726 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:38:31.726 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":72,"fsid":"70142e68-c7e6-11ee-95b8-87774f69a715","created":"2024-02-10T07:32:26.906892+0000","modified":"2024-02-10T07:38:28.768097+0000","last_up_change":"2024-02-10T07:37:43.894235+0000","last_in_change":"2024-02-10T07:37:26.615461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T07:35:41.128169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-10T07:37:56.085687+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-10T07:37:56.748832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-10T07:37:58.456399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-10T07:38:00.407577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-10T07:38:02.481040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21f7c994-f25f-4bfe-ba6b-bcb06c637113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":3723357830}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":3723357830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":3723357830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":3723357830}]},"public_addr":"172.21.15.70:6801/3723357830","cluster_addr":"172.21.15.70:6802/3723357830","heartbeat_back_addr":"172.21.15.70:6804/3723357830","heartbeat_front_addr":"172.21.15.70:6803/3723357830","state":["exists","up"]},{"osd":1,"uuid":"a4164b54-8dde-4071-846d-959548c7fbe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":918939980}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":918939980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":918939980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":918939980}]},"public_addr":"172.21.15.70:6805/918939980","cluster_addr":"172.21.15.70:6806/918939980","heartbeat_back_addr":"172.21.15.70:6808/918939980","heartbeat_front_addr":"172.21.15.70:6807/918939980","state":["exists","up"]},{"osd":2,"uuid":"b5ea7978-5f0f-4ba4-a881-ed48efdd57d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":3606696222}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":3606696222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":3606696222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":3606696222}]},"public_addr":"172.21.15.70:6809/3606696222","cluster_addr":"172.21.15.70:6810/3606696222","heartbeat_back_addr":"172.21.15.70:6812/3606696222","heartbeat_front_addr":"172.21.15.70:6811/3606696222","state":["exists","up"]},{"osd":3,"uuid":"5953e938-5967-420e-9262-2c2ba2ab3bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":3512826399}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":3512826399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6816","nonce":3512826399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":3512826399}]},"public_addr":"172.21.15.70:6813/3512826399","cluster_addr":"172.21.15.70:6814/3512826399","heartbeat_back_addr":"172.21.15.70:6816/3512826399","heartbeat_front_addr":"172.21.15.70:6815/3512826399","state":["exists","up"]},{"osd":4,"uuid":"f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":1901321685}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":1901321685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":1901321685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":1901321685}]},"public_addr":"172.21.15.155:6800/1901321685","cluster_addr":"172.21.15.155:6801/1901321685","heartbeat_back_addr":"172.21.15.155:6803/1901321685","heartbeat_front_addr":"172.21.15.155:6802/1901321685","state":["exists","up"]},{"osd":5,"uuid":"4f131619-da44-4168-a436-8a7525448564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":969979851}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":969979851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":969979851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":969979851}]},"public_addr":"172.21.15.155:6804/969979851","cluster_addr":"172.21.15.155:6805/969979851","heartbeat_back_addr":"172.21.15.155:6807/969979851","heartbeat_front_addr":"172.21.15.155:6806/969979851","state":["exists","up"]},{"osd":6,"uuid":"4d2ab766-9537-43a1-8f08-621786e62715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":3655535506}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":3655535506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":3655535506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":3655535506}]},"public_addr":"172.21.15.155:6808/3655535506","cluster_addr":"172.21.15.155:6809/3655535506","heartbeat_back_addr":"172.21.15.155:6811/3655535506","heartbeat_front_addr":"172.21.15.155:6810/3655535506","state":["exists","up"]},{"osd":7,"uuid":"2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":2127871237}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":2127871237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":2127871237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":2127871237}]},"public_addr":"172.21.15.155:6812/2127871237","cluster_addr":"172.21.15.155:6813/2127871237","heartbeat_back_addr":"172.21.15.155:6815/2127871237","heartbeat_front_addr":"172.21.15.155:6814/2127871237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:34:48.078516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:35:13.167179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:35:37.892306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:03.973193+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:27.390006+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:52.444572+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:37:17.537509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:37:42.663953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6800/1990687880":"2024-02-11T07:33:26.790782+0000","172.21.15.70:6800/1444653888":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/2812082303":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/3407186454":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/3824912336":"2024-02-11T07:33:26.790782+0000","172.21.15.70:0/3785821604":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/664263038":"2024-02-11T07:33:26.790782+0000","172.21.15.70:0/3841311618":"2024-02-11T07:33:26.790782+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T07:38:31.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.728+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 >> v1:172.21.15.70:6800/3858935040 conn(0x7f63a006ead0 legacy=0x7f63a0070f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T07:38:31.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.728+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 >> v1:172.21.15.70:6789/0 conn(0x7f63b410ab20 legacy=0x7f63b41a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:31.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.728+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 shutdown_connections 2024-02-10T07:38:31.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.728+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 >> 172.21.15.70:0/2893648946 conn(0x7f63b40fc4c0 msgr2=0x7f63b40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:31.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.728+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 shutdown_connections 2024-02-10T07:38:31.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:31.728+0000 7f63bb5f9700 1 -- 172.21.15.70:0/2893648946 wait complete. 2024-02-10T07:38:31.991 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:31.780+0000 7fcc9dbff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:38:31.991 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:31 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:31.875+0000 7fcc9dbff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:38:32.005 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:31.720+0000 7fa3e1e16200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:38:32.005 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:31.814+0000 7fa3e1e16200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:38:32.005 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:31.956+0000 7fa3e1e16200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:38:32.400 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:32.052+0000 7fa3e1e16200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:38:32.400 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:32.134+0000 7fa3e1e16200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:38:32.400 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:32.399+0000 7fa3e1e16200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:38:32.424 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.019+0000 7fcc9dbff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:38:32.424 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.115+0000 7fcc9dbff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:38:32.424 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.200+0000 7fcc9dbff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:38:32.549 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-10T07:38:32.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T07:38:32.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2893648946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:38:32.676 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2893648946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:38:32.676 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.468+0000 7fcc9dbff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:38:32.676 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.583+0000 7fcc9dbff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:38:32.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2893648946' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:38:32.693 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:32.510+0000 7fa3e1e16200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:38:32.693 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:32.597+0000 7fa3e1e16200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:38:32.693 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:32.692+0000 7fa3e1e16200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:38:32.798 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:32.937 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.675+0000 7fcc9dbff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:38:32.937 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.780+0000 7fcc9dbff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:38:33.005 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:32.870+0000 7fa3e1e16200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:38:33.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:32.972+0000 7fcc9dbff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:38:34.216 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:33 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:33.950+0000 7fcc9dbff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:38:34.216 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:34.086+0000 7fcc9dbff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:38:34.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:33.834+0000 7fa3e1e16200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:38:34.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:33.965+0000 7fa3e1e16200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:38:34.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.427+0000 7f87ff572700 1 -- 172.21.15.70:0/3340933696 >> v1:172.21.15.70:6789/0 conn(0x7f87f810d430 legacy=0x7f87f810f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:34.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.427+0000 7f87ff572700 1 -- 172.21.15.70:0/3340933696 shutdown_connections 2024-02-10T07:38:34.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.427+0000 7f87ff572700 1 -- 172.21.15.70:0/3340933696 >> 172.21.15.70:0/3340933696 conn(0x7f87f80fc4c0 msgr2=0x7f87f80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:34.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.427+0000 7f87ff572700 1 -- 172.21.15.70:0/3340933696 shutdown_connections 2024-02-10T07:38:34.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.427+0000 7f87ff572700 1 -- 172.21.15.70:0/3340933696 wait complete. 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ff572700 1 Processor -- start 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ff572700 1 -- start start 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ff572700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87f81a6a60 con 0x7f87f8109860 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ff572700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87f81a7c40 con 0x7f87f81069b0 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ff572700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f87f81a8e40 con 0x7f87f810d430 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87fcb0d700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f87f8109860 0x7f87f81a17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41978/0 (socket says 172.21.15.70:41978) 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87fcb0d700 1 -- 172.21.15.70:0/3192588029 learned_addr learned my addr 172.21.15.70:0/3192588029 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5614175 0 0) 0x7f87f81a6a60 con 0x7f87f8109860 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e40035b0 con 0x7f87f8109860 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620153638 0 0) 0x7f87f81a8e40 con 0x7f87f810d430 2024-02-10T07:38:34.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f81a6a60 con 0x7f87f810d430 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187965677 0 0) 0x7f87f81a7c40 con 0x7f87f81069b0 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f81a8e40 con 0x7f87f81069b0 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 193623634 0 0) 0x7f87e40035b0 con 0x7f87f8109860 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87f81a7c40 con 0x7f87f8109860 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f87e8001930 con 0x7f87f8109860 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2384027008 0 0) 0x7f87f81a6a60 con 0x7f87f810d430 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87e40035b0 con 0x7f87f810d430 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f87f0004730 con 0x7f87f810d430 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4017480038 0 0) 0x7f87f81a8e40 con 0x7f87f81069b0 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.428+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87f81a6a60 con 0x7f87f81069b0 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f87f4001830 con 0x7f87f81069b0 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3588957665 0 0) 0x7f87f81a6a60 con 0x7f87f81069b0 2024-02-10T07:38:34.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 >> v1:172.21.15.70:6790/0 conn(0x7f87f810d430 legacy=0x7f87f81a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 >> v1:172.21.15.70:6789/0 conn(0x7f87f8109860 legacy=0x7f87f81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f81aa040 con 0x7f87f81069b0 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f81a9030 con 0x7f87f81069b0 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f81a94b0 con 0x7f87f81069b0 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f87f4003b10 con 0x7f87f81069b0 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.429+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f87f40049f0 con 0x7f87f81069b0 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.430+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc005150 con 0x7f87f81069b0 2024-02-10T07:38:34.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.430+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 17) v1 ==== 91436+0+0 (unknown 2333988677 0 0) 0x7f87f4003680 con 0x7f87f81069b0 2024-02-10T07:38:34.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.431+0000 7f87fcb0d700 1 -- 172.21.15.70:0/3192588029 >> v1:172.21.15.70:6800/3858935040 conn(0x7f87e406e990 legacy=0x7f87e4070e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/3858935040 2024-02-10T07:38:34.433 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.431+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 2587513750 0 0) 0x7f87f4087650 con 0x7f87f81069b0 2024-02-10T07:38:34.435 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.435+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f87f4050cd0 con 0x7f87f81069b0 2024-02-10T07:38:34.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:34.446+0000 7fcc9dbff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:38:34.594 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:34.304+0000 7fa3e1e16200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:38:34.594 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:34.395+0000 7fa3e1e16200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:38:34.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.632+0000 7f87fcb0d700 1 -- 172.21.15.70:0/3192588029 >> v1:172.21.15.70:6800/3858935040 conn(0x7f87e406e990 legacy=0x7f87e4070e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6800/3858935040 2024-02-10T07:38:34.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.637+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87dc005400 con 0x7f87f81069b0 2024-02-10T07:38:34.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.639+0000 7f87ee7fc700 1 -- 172.21.15.70:0/3192588029 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+20671 (unknown 1686360691 0 834537700) 0x7f87f4052c00 con 0x7f87f81069b0 2024-02-10T07:38:34.639 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:38:34.639 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":72,"fsid":"70142e68-c7e6-11ee-95b8-87774f69a715","created":"2024-02-10T07:32:26.906892+0000","modified":"2024-02-10T07:38:28.768097+0000","last_up_change":"2024-02-10T07:37:43.894235+0000","last_in_change":"2024-02-10T07:37:26.615461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T07:35:41.128169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-10T07:37:56.085687+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-10T07:37:56.748832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-10T07:37:58.456399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-10T07:38:00.407577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-10T07:38:02.481040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"21f7c994-f25f-4bfe-ba6b-bcb06c637113","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":3723357830}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":3723357830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":3723357830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":3723357830}]},"public_addr":"172.21.15.70:6801/3723357830","cluster_addr":"172.21.15.70:6802/3723357830","heartbeat_back_addr":"172.21.15.70:6804/3723357830","heartbeat_front_addr":"172.21.15.70:6803/3723357830","state":["exists","up"]},{"osd":1,"uuid":"a4164b54-8dde-4071-846d-959548c7fbe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":918939980}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":918939980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":918939980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":918939980}]},"public_addr":"172.21.15.70:6805/918939980","cluster_addr":"172.21.15.70:6806/918939980","heartbeat_back_addr":"172.21.15.70:6808/918939980","heartbeat_front_addr":"172.21.15.70:6807/918939980","state":["exists","up"]},{"osd":2,"uuid":"b5ea7978-5f0f-4ba4-a881-ed48efdd57d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":3606696222}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":3606696222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":3606696222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":3606696222}]},"public_addr":"172.21.15.70:6809/3606696222","cluster_addr":"172.21.15.70:6810/3606696222","heartbeat_back_addr":"172.21.15.70:6812/3606696222","heartbeat_front_addr":"172.21.15.70:6811/3606696222","state":["exists","up"]},{"osd":3,"uuid":"5953e938-5967-420e-9262-2c2ba2ab3bd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":3512826399}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":3512826399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6816","nonce":3512826399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":3512826399}]},"public_addr":"172.21.15.70:6813/3512826399","cluster_addr":"172.21.15.70:6814/3512826399","heartbeat_back_addr":"172.21.15.70:6816/3512826399","heartbeat_front_addr":"172.21.15.70:6815/3512826399","state":["exists","up"]},{"osd":4,"uuid":"f175b8a6-e50d-44e6-b5e5-3f5c3fbb3e51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":1901321685}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":1901321685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":1901321685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":1901321685}]},"public_addr":"172.21.15.155:6800/1901321685","cluster_addr":"172.21.15.155:6801/1901321685","heartbeat_back_addr":"172.21.15.155:6803/1901321685","heartbeat_front_addr":"172.21.15.155:6802/1901321685","state":["exists","up"]},{"osd":5,"uuid":"4f131619-da44-4168-a436-8a7525448564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":969979851}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":969979851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":969979851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":969979851}]},"public_addr":"172.21.15.155:6804/969979851","cluster_addr":"172.21.15.155:6805/969979851","heartbeat_back_addr":"172.21.15.155:6807/969979851","heartbeat_front_addr":"172.21.15.155:6806/969979851","state":["exists","up"]},{"osd":6,"uuid":"4d2ab766-9537-43a1-8f08-621786e62715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":3655535506}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":3655535506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":3655535506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":3655535506}]},"public_addr":"172.21.15.155:6808/3655535506","cluster_addr":"172.21.15.155:6809/3655535506","heartbeat_back_addr":"172.21.15.155:6811/3655535506","heartbeat_front_addr":"172.21.15.155:6810/3655535506","state":["exists","up"]},{"osd":7,"uuid":"2f81bbbf-6fc0-4482-bb9b-bdad400dd0f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":2127871237}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":2127871237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":2127871237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":2127871237}]},"public_addr":"172.21.15.155:6812/2127871237","cluster_addr":"172.21.15.155:6813/2127871237","heartbeat_back_addr":"172.21.15.155:6815/2127871237","heartbeat_front_addr":"172.21.15.155:6814/2127871237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:34:48.078516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:35:13.167179+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:35:37.892306+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:03.973193+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:27.390006+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:36:52.444572+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:37:17.537509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:37:42.663953+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6800/1990687880":"2024-02-11T07:33:26.790782+0000","172.21.15.70:6800/1444653888":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/2812082303":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/3407186454":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/3824912336":"2024-02-11T07:33:26.790782+0000","172.21.15.70:0/3785821604":"2024-02-11T07:33:00.774796+0000","172.21.15.70:0/664263038":"2024-02-11T07:33:26.790782+0000","172.21.15.70:0/3841311618":"2024-02-11T07:33:26.790782+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T07:38:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.641+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 >> v1:172.21.15.70:6800/3858935040 conn(0x7f87e406e990 legacy=0x7f87e4070e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T07:38:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.641+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 >> v1:172.21.15.155:6789/0 conn(0x7f87f81069b0 legacy=0x7f87f810cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.641+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 shutdown_connections 2024-02-10T07:38:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.641+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 >> 172.21.15.70:0/3192588029 conn(0x7f87f80fc4c0 msgr2=0x7f87f81009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.641+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 shutdown_connections 2024-02-10T07:38:34.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:34.641+0000 7f87ff572700 1 -- 172.21.15.70:0/3192588029 wait complete. 2024-02-10T07:38:34.935 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3192588029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:38:34.935 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3192588029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:38:34.935 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:34.540+0000 7fcc9dbff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:38:34.935 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:34.745+0000 7fcc9dbff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:38:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3192588029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:38:35.005 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:34.594+0000 7fa3e1e16200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:38:35.005 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:34.781+0000 7fa3e1e16200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:38:35.223 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:34.934+0000 7fcc9dbff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:38:35.331 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:35 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:35.069+0000 7fa3e1e16200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:38:35.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-10T07:38:35.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-10T07:38:35.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-10T07:38:35.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-10T07:38:35.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-10T07:38:35.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-10T07:38:35.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-10T07:38:35.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-10T07:38:35.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:35.223+0000 7fcc9dbff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:38:35.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:35 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:35.331+0000 7fa3e1e16200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:38:35.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:35 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:35.568+0000 7fa3e1e16200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:38:35.879 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:35.947 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:35.556+0000 7fcc9dbff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:38:35.947 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:35.946+0000 7fcc9dbff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:38:35.947 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:36.580 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:36.656 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:36 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:36.376+0000 7fa3e1e16200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:38:36.991 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:36 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:36.804+0000 7fcc9dbff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:38:37.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:36 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:36.956+0000 7fa3e1e16200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:38:37.255 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:37.056+0000 7fa3e1e16200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:38:37.655 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:37.403+0000 7fcc9dbff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:38:37.656 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:37.504+0000 7fcc9dbff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:38:37.668 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:37.274+0000 7fa3e1e16200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:38:37.668 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: [10/Feb/2024:07:38:37] ENGINE Bus STARTING 2024-02-10T07:38:37.668 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: CherryPy Checker: 2024-02-10T07:38:37.668 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: The Application mounted at '' has an empty config. 2024-02-10T07:38:37.668 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: 2024-02-10T07:38:37.668 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: [10/Feb/2024:07:38:37] ENGINE Serving on http://:::9283 2024-02-10T07:38:37.668 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x[34536]: [10/Feb/2024:07:38:37] ENGINE Bus STARTED 2024-02-10T07:38:37.919 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: Standby manager daemon x restarted 2024-02-10T07:38:37.919 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: Standby manager daemon x started 2024-02-10T07:38:37.919 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:38:37.919 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:38:37.919 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:38:37.919 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: Active manager daemon y restarted 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: Activating manager daemon y 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: osdmap e73: 8 total, 8 up, 8 in 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: mgrmap e18: y(active, starting, since 0.00713861s), standbys: x 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:38:37.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:38:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:38:37.922 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:37.738+0000 7fcc9dbff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:38:37.922 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:38:37] ENGINE Bus STARTING 2024-02-10T07:38:37.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: Standby manager daemon x restarted 2024-02-10T07:38:37.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: Standby manager daemon x started 2024-02-10T07:38:37.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:38:37.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: Active manager daemon y restarted 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: Activating manager daemon y 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: osdmap e73: 8 total, 8 up, 8 in 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: mgrmap e18: y(active, starting, since 0.00713861s), standbys: x 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:38:37.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:38:37.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:38:37.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:38:37.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:38:37.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:38:37.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:38:37.925 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: Standby manager daemon x restarted 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: Standby manager daemon x started 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.? v1:172.21.15.155:0/1658573191' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: Active manager daemon y restarted 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: Activating manager daemon y 2024-02-10T07:38:37.926 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: osdmap e73: 8 total, 8 up, 8 in 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: mgrmap e18: y(active, starting, since 0.00713861s), standbys: x 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:38:37.927 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T07:38:37.928 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T07:38:37.928 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T07:38:37.928 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T07:38:37.928 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T07:38:37.928 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:38:37.928 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:38:37.928 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:38:38.175 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: CherryPy Checker: 2024-02-10T07:38:38.175 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: The Application mounted at '' has an empty config. 2024-02-10T07:38:38.175 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: 2024-02-10T07:38:38.176 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:38:37] ENGINE Serving on http://:::9283 2024-02-10T07:38:38.176 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:38:37] ENGINE Bus STARTED 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: Manager daemon y is now available 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: [10/Feb/2024:07:38:38] ENGINE Bus STARTING 2024-02-10T07:38:38.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: [10/Feb/2024:07:38:38] ENGINE Serving on http://172.21.15.70:8765 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: [10/Feb/2024:07:38:38] ENGINE Serving on https://172.21.15.70:7150 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[39923]: [10/Feb/2024:07:38:38] ENGINE Bus STARTED 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: Manager daemon y is now available 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:38:38.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:38:38.826 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: [10/Feb/2024:07:38:38] ENGINE Bus STARTING 2024-02-10T07:38:38.826 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: [10/Feb/2024:07:38:38] ENGINE Serving on http://172.21.15.70:8765 2024-02-10T07:38:38.826 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: [10/Feb/2024:07:38:38] ENGINE Serving on https://172.21.15.70:7150 2024-02-10T07:38:38.826 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:38 smithi070 ceph-mon[29548]: [10/Feb/2024:07:38:38] ENGINE Bus STARTED 2024-02-10T07:38:38.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.924+0000 7fd1385ed700 1 -- 172.21.15.70:0/3742809225 >> v1:172.21.15.70:6790/0 conn(0x7fd130109870 legacy=0x7fd13010bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.924+0000 7fd1385ed700 1 -- 172.21.15.70:0/3742809225 shutdown_connections 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.924+0000 7fd1385ed700 1 -- 172.21.15.70:0/3742809225 >> 172.21.15.70:0/3742809225 conn(0x7fd1300fc4b0 msgr2=0x7fd1300fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.924+0000 7fd1385ed700 1 -- 172.21.15.70:0/3742809225 shutdown_connections 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.924+0000 7fd1385ed700 1 -- 172.21.15.70:0/3742809225 wait complete. 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1385ed700 1 Processor -- start 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1385ed700 1 -- start start 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1385ed700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd1301a8b30 con 0x7fd1301069c0 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1385ed700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd1301a9d10 con 0x7fd130109870 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1385ed700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd1301aaef0 con 0x7fd13010d440 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd136389700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd1301069c0 0x7fd13010c930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33358/0 (socket says 172.21.15.70:33358) 2024-02-10T07:38:38.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd136389700 1 -- 172.21.15.70:0/2799038244 learned_addr learned my addr 172.21.15.70:0/2799038244 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:38.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065209481 0 0) 0x7fd1301a8b30 con 0x7fd1301069c0 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd11c0035b0 con 0x7fd1301069c0 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271425371 0 0) 0x7fd1301aaef0 con 0x7fd13010d440 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.925+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1301a8b30 con 0x7fd13010d440 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519265654 0 0) 0x7fd1301a9d10 con 0x7fd130109870 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1301aaef0 con 0x7fd130109870 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 736076046 0 0) 0x7fd11c0035b0 con 0x7fd1301069c0 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1301a9d10 con 0x7fd1301069c0 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd1200019d0 con 0x7fd1301069c0 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2159228946 0 0) 0x7fd1301a8b30 con 0x7fd13010d440 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd11c0035b0 con 0x7fd13010d440 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1443876310 0 0) 0x7fd1301aaef0 con 0x7fd130109870 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1301a8b30 con 0x7fd130109870 2024-02-10T07:38:38.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd128001bb0 con 0x7fd13010d440 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd12c0043a0 con 0x7fd130109870 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1618458177 0 0) 0x7fd1301a9d10 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 >> v1:172.21.15.70:6790/0 conn(0x7fd13010d440 legacy=0x7fd1301a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 >> v1:172.21.15.155:6789/0 conn(0x7fd130109870 legacy=0x7fd1301a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1301ac0f0 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1301ab0e0 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1301ab560 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fd120003b10 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.926+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd120005bf0 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.927+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd114000f80 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.928+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 19) v1 ==== 91180+0+0 (unknown 1764454933 0 0) 0x7fd120003680 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.928+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fd120088150 con 0x7fd1301069c0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.928+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6809/3606696222 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd11c07ccd0 con 0x7fd11c078fc0 2024-02-10T07:38:38.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.928+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7fd1200885b0 con 0x7fd1301069c0 2024-02-10T07:38:38.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.929+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== osd.2 v1:172.21.15.70:6809/3606696222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fd11c07ccd0 con 0x7fd11c078fc0 2024-02-10T07:38:38.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.945+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 --> v1:172.21.15.70:6809/3606696222 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd114002c40 con 0x7fd11c078fc0 2024-02-10T07:38:38.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1277fe700 1 -- 172.21.15.70:0/2799038244 <== osd.2 v1:172.21.15.70:6809/3606696222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2431347361) 0x7fd114002c40 con 0x7fd11c078fc0 2024-02-10T07:38:38.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 >> v1:172.21.15.70:6809/3606696222 conn(0x7fd11c078fc0 legacy=0x7fd11c07b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:38.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 >> v1:172.21.15.70:6800/4101821536 conn(0x7fd11c06e850 legacy=0x7fd11c070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:38.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 >> v1:172.21.15.70:6789/0 conn(0x7fd1301069c0 legacy=0x7fd13010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:38.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd136b8a700 1 -- 172.21.15.70:0/2799038244 reap_dead start 2024-02-10T07:38:38.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 shutdown_connections 2024-02-10T07:38:38.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 >> 172.21.15.70:0/2799038244 conn(0x7fd1300fc4b0 msgr2=0x7fd130110940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:38.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 shutdown_connections 2024-02-10T07:38:38.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:38.946+0000 7fd1385ed700 1 -- 172.21.15.70:0/2799038244 wait complete. 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: Manager daemon y is now available 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: [10/Feb/2024:07:38:38] ENGINE Bus STARTING 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: [10/Feb/2024:07:38:38] ENGINE Serving on http://172.21.15.70:8765 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: [10/Feb/2024:07:38:38] ENGINE Serving on https://172.21.15.70:7150 2024-02-10T07:38:39.051 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:38 smithi155 ceph-mon[32688]: [10/Feb/2024:07:38:38] ENGINE Bus STARTED 2024-02-10T07:38:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:39 smithi155 ceph-mon[32688]: mgrmap e19: y(active, since 1.03158s), standbys: x 2024-02-10T07:38:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:39 smithi155 ceph-mon[32688]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:40.087 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:39 smithi070 ceph-mon[39923]: mgrmap e19: y(active, since 1.03158s), standbys: x 2024-02-10T07:38:40.087 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:40.087 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:39 smithi070 ceph-mon[39923]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:40.088 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:39 smithi070 ceph-mon[29548]: mgrmap e19: y(active, since 1.03158s), standbys: x 2024-02-10T07:38:40.088 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:40.088 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:39 smithi070 ceph-mon[29548]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:41.131 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:40 smithi070 ceph-mon[29548]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:41.131 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:40 smithi070 ceph-mon[29548]: mgrmap e20: y(active, since 2s), standbys: x 2024-02-10T07:38:41.131 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:40 smithi070 ceph-mon[39923]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:41.131 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:40 smithi070 ceph-mon[39923]: mgrmap e20: y(active, since 2s), standbys: x 2024-02-10T07:38:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:40 smithi155 ceph-mon[32688]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:40 smithi155 ceph-mon[32688]: mgrmap e20: y(active, since 2s), standbys: x 2024-02-10T07:38:42.140 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.141 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.141 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.141 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:38:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:38:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.243+0000 7f112244d700 1 -- 172.21.15.70:0/3906119392 >> v1:172.21.15.70:6790/0 conn(0x7f111c10d430 legacy=0x7f111c10f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.243+0000 7f112244d700 1 -- 172.21.15.70:0/3906119392 shutdown_connections 2024-02-10T07:38:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.243+0000 7f112244d700 1 -- 172.21.15.70:0/3906119392 >> 172.21.15.70:0/3906119392 conn(0x7f111c0fc4c0 msgr2=0x7f111c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.243+0000 7f112244d700 1 -- 172.21.15.70:0/3906119392 shutdown_connections 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.243+0000 7f112244d700 1 -- 172.21.15.70:0/3906119392 wait complete. 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f112244d700 1 Processor -- start 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f112244d700 1 -- start start 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f112244d700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f111c1a5830 con 0x7f111c10d430 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f112244d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f111c1a6a10 con 0x7f111c109860 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f112244d700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f111c1a7c10 con 0x7f111c1069b0 2024-02-10T07:38:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f111bfff700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f111c1069b0 0x7f111c10cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:58398/0 (socket says 172.21.15.70:58398) 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f111bfff700 1 -- 172.21.15.70:0/3510582019 learned_addr learned my addr 172.21.15.70:0/3510582019 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.244+0000 7f11209ea700 1 --1- 172.21.15.70:0/3510582019 >> v1:172.21.15.70:6789/0 conn(0x7f111c10d430 0x7f111c1a3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33398/0 (socket says 172.21.15.70:33398) 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742512827 0 0) 0x7f111c1a7c10 con 0x7f111c1069b0 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11040035b0 con 0x7f111c1069b0 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81897988 0 0) 0x7f111c1a5830 con 0x7f111c10d430 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f111c1a7c10 con 0x7f111c10d430 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901761785 0 0) 0x7f111c1a6a10 con 0x7f111c109860 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f111c1a5830 con 0x7f111c109860 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2648193912 0 0) 0x7f11040035b0 con 0x7f111c1069b0 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f111c1a6a10 con 0x7f111c1069b0 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f110c0019d0 con 0x7f111c1069b0 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1231412236 0 0) 0x7f111c1a7c10 con 0x7f111c10d430 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f11040035b0 con 0x7f111c10d430 2024-02-10T07:38:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 212388535 0 0) 0x7f111c1a5830 con 0x7f111c109860 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f111c1a7c10 con 0x7f111c109860 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f11140047e0 con 0x7f111c10d430 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f11100017f0 con 0x7f111c109860 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1978663376 0 0) 0x7f111c1a6a10 con 0x7f111c1069b0 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 >> v1:172.21.15.155:6789/0 conn(0x7f111c109860 legacy=0x7f111c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 >> v1:172.21.15.70:6789/0 conn(0x7f111c10d430 legacy=0x7f111c1a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c1a8e10 con 0x7f111c1069b0 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c1a5a20 con 0x7f111c1069b0 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c1a5ea0 con 0x7f111c1069b0 2024-02-10T07:38:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f110c003b10 con 0x7f111c1069b0 2024-02-10T07:38:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.245+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f110c005bc0 con 0x7f111c1069b0 2024-02-10T07:38:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.246+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.70:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1108000f80 con 0x7f111c1069b0 2024-02-10T07:38:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.247+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 20) v1 ==== 91372+0+0 (unknown 3703994306 0 0) 0x7f110c003680 con 0x7f111c1069b0 2024-02-10T07:38:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.247+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f110c0887c0 con 0x7f111c1069b0 2024-02-10T07:38:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.247+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.155:6804/969979851 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f110407f2b0 con 0x7f110407b5a0 2024-02-10T07:38:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.247+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f110c088b10 con 0x7f111c1069b0 2024-02-10T07:38:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.249+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== osd.5 v1:172.21.15.155:6804/969979851 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f110407f2b0 con 0x7f110407b5a0 2024-02-10T07:38:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.265+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 --> v1:172.21.15.155:6804/969979851 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1108002c40 con 0x7f110407b5a0 2024-02-10T07:38:42.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.265+0000 7f11197fa700 1 -- 172.21.15.70:0/3510582019 <== osd.5 v1:172.21.15.155:6804/969979851 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1238763653) 0x7f1108002c40 con 0x7f110407b5a0 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 >> v1:172.21.15.155:6804/969979851 conn(0x7f110407b5a0 legacy=0x7f110407da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 >> v1:172.21.15.70:6800/4101821536 conn(0x7f1104070e40 legacy=0x7f1104073300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 >> v1:172.21.15.70:6790/0 conn(0x7f111c1069b0 legacy=0x7f111c10cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f11209ea700 1 -- 172.21.15.70:0/3510582019 reap_dead start 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 shutdown_connections 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 >> 172.21.15.70:0/3510582019 conn(0x7f111c0fc4c0 msgr2=0x7f111c110950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 shutdown_connections 2024-02-10T07:38:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.266+0000 7f112244d700 1 -- 172.21.15.70:0/3510582019 wait complete. 2024-02-10T07:38:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.988+0000 7f1f39d1a700 1 -- 172.21.15.70:0/25119941 >> v1:172.21.15.70:6790/0 conn(0x7f1f3410a9a0 legacy=0x7f1f3410cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.988+0000 7f1f39d1a700 1 -- 172.21.15.70:0/25119941 shutdown_connections 2024-02-10T07:38:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.988+0000 7f1f39d1a700 1 -- 172.21.15.70:0/25119941 >> 172.21.15.70:0/25119941 conn(0x7f1f340fd630 msgr2=0x7f1f340ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.988+0000 7f1f39d1a700 1 -- 172.21.15.70:0/25119941 shutdown_connections 2024-02-10T07:38:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.988+0000 7f1f39d1a700 1 -- 172.21.15.70:0/25119941 wait complete. 2024-02-10T07:38:42.990 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.989+0000 7f1f39d1a700 1 Processor -- start 2024-02-10T07:38:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.989+0000 7f1f39d1a700 1 -- start start 2024-02-10T07:38:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.989+0000 7f1f39d1a700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1f34107380 con 0x7f1f34107bf0 2024-02-10T07:38:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.989+0000 7f1f39d1a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1f34107550 con 0x7f1f3410e5d0 2024-02-10T07:38:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.989+0000 7f1f39d1a700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1f341a9110 con 0x7f1f3410a9a0 2024-02-10T07:38:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f32ffd700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f1f3410a9a0 0x7f1f341048e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:58426/0 (socket says 172.21.15.70:58426) 2024-02-10T07:38:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f32ffd700 1 -- 172.21.15.70:0/3294993079 learned_addr learned my addr 172.21.15.70:0/3294993079 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:42.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f337fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1f34107bf0 0x7f1f341041d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33410/0 (socket says 172.21.15.70:33410) 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102894523 0 0) 0x7f1f341a9110 con 0x7f1f3410a9a0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f1c0035b0 con 0x7f1f3410a9a0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878335051 0 0) 0x7f1f34107380 con 0x7f1f34107bf0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f341a9110 con 0x7f1f34107bf0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378130276 0 0) 0x7f1f34107550 con 0x7f1f3410e5d0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f34107380 con 0x7f1f3410e5d0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4082811896 0 0) 0x7f1f1c0035b0 con 0x7f1f3410a9a0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1f34107550 con 0x7f1f3410a9a0 2024-02-10T07:38:42.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1f28001500 con 0x7f1f3410a9a0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 810996758 0 0) 0x7f1f341a9110 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1f1c0035b0 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1f24001830 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2965928658 0 0) 0x7f1f34107380 con 0x7f1f3410e5d0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1f341a9110 con 0x7f1f3410e5d0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1f2c0013f0 con 0x7f1f3410e5d0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4242338775 0 0) 0x7f1f1c0035b0 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 >> v1:172.21.15.70:6790/0 conn(0x7f1f3410a9a0 legacy=0x7f1f341048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 >> v1:172.21.15.155:6789/0 conn(0x7f1f3410e5d0 legacy=0x7f1f341a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f341aa2f0 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f341a9300 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.990+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f341a9780 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.991+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f1f24001d20 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.991+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1f24005a90 con 0x7f1f34107bf0 2024-02-10T07:38:42.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.991+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f20000f80 con 0x7f1f34107bf0 2024-02-10T07:38:42.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.992+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f1f24003680 con 0x7f1f34107bf0 2024-02-10T07:38:42.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.992+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f1f24088730 con 0x7f1f34107bf0 2024-02-10T07:38:42.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.993+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.155:6808/3655535506 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f1c07cde0 con 0x7f1f1c0790d0 2024-02-10T07:38:42.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.993+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f1f24088ba0 con 0x7f1f34107bf0 2024-02-10T07:38:42.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:42.994+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== osd.6 v1:172.21.15.155:6808/3655535506 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f1f1c07cde0 con 0x7f1f1c0790d0 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi155 to 3760M 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[39923]: mgrmap e21: y(active, since 5s), standbys: x 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi155 to 3760M 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:42.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:42 smithi070 ceph-mon[29548]: mgrmap e21: y(active, since 5s), standbys: x 2024-02-10T07:38:43.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.014+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 --> v1:172.21.15.155:6808/3655535506 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f20002c40 con 0x7f1f1c0790d0 2024-02-10T07:38:43.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f30ff9700 1 -- 172.21.15.70:0/3294993079 <== osd.6 v1:172.21.15.155:6808/3655535506 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3377452340) 0x7f1f20002c40 con 0x7f1f1c0790d0 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 >> v1:172.21.15.155:6808/3655535506 conn(0x7f1f1c0790d0 legacy=0x7f1f1c07b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 >> v1:172.21.15.70:6800/4101821536 conn(0x7f1f1c06e9f0 legacy=0x7f1f1c070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 >> v1:172.21.15.70:6789/0 conn(0x7f1f34107bf0 legacy=0x7f1f341041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f33fff700 1 -- 172.21.15.70:0/3294993079 reap_dead start 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 shutdown_connections 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 >> 172.21.15.70:0/3294993079 conn(0x7f1f340fd630 msgr2=0x7f1f34100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 shutdown_connections 2024-02-10T07:38:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:43.015+0000 7f1f39d1a700 1 -- 172.21.15.70:0/3294993079 wait complete. 2024-02-10T07:38:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:38:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi155 to 3760M 2024-02-10T07:38:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:38:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:42 smithi155 ceph-mon[32688]: mgrmap e21: y(active, since 5s), standbys: x 2024-02-10T07:38:44.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.627+0000 7fbea23eb700 1 -- 172.21.15.70:0/451531147 >> v1:172.21.15.70:6790/0 conn(0x7fbe9c107b60 legacy=0x7fbe9c107f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:44.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.627+0000 7fbea23eb700 1 -- 172.21.15.70:0/451531147 shutdown_connections 2024-02-10T07:38:44.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.627+0000 7fbea23eb700 1 -- 172.21.15.70:0/451531147 >> 172.21.15.70:0/451531147 conn(0x7fbe9c0fd600 msgr2=0x7fbe9c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:44.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.627+0000 7fbea23eb700 1 -- 172.21.15.70:0/451531147 shutdown_connections 2024-02-10T07:38:44.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.627+0000 7fbea23eb700 1 -- 172.21.15.70:0/451531147 wait complete. 2024-02-10T07:38:44.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.628+0000 7fbea23eb700 1 Processor -- start 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.628+0000 7fbea23eb700 1 -- start start 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.628+0000 7fbea23eb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbe9c1a8c10 con 0x7fbe9c10e550 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.628+0000 7fbea23eb700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbe9c1a9df0 con 0x7fbe9c10a980 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.628+0000 7fbea23eb700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbe9c1aafd0 con 0x7fbe9c107b60 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.628+0000 7fbe9bfff700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fbe9c107b60 0x7fbe9c075970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:58442/0 (socket says 172.21.15.70:58442) 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.628+0000 7fbe9bfff700 1 -- 172.21.15.70:0/116871625 learned_addr learned my addr 172.21.15.70:0/116871625 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936767460 0 0) 0x7fbe9c1a8c10 con 0x7fbe9c10e550 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe840035b0 con 0x7fbe9c10e550 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107915279 0 0) 0x7fbe9c1aafd0 con 0x7fbe9c107b60 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe9c1a8c10 con 0x7fbe9c107b60 2024-02-10T07:38:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124649737 0 0) 0x7fbe9c1a9df0 con 0x7fbe9c10a980 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe9c1aafd0 con 0x7fbe9c10a980 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 816960641 0 0) 0x7fbe840035b0 con 0x7fbe9c10e550 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbe9c1a9df0 con 0x7fbe9c10e550 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fbe94001470 con 0x7fbe9c10e550 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3024533072 0 0) 0x7fbe9c1a8c10 con 0x7fbe9c107b60 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbe840035b0 con 0x7fbe9c107b60 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fbe8c004430 con 0x7fbe9c107b60 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3844085751 0 0) 0x7fbe9c1a9df0 con 0x7fbe9c10e550 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 >> v1:172.21.15.70:6790/0 conn(0x7fbe9c107b60 legacy=0x7fbe9c075970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 >> v1:172.21.15.155:6789/0 conn(0x7fbe9c10a980 legacy=0x7fbe9c1a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c1ac1d0 con 0x7fbe9c10e550 2024-02-10T07:38:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c1a8e00 con 0x7fbe9c10e550 2024-02-10T07:38:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c1a9280 con 0x7fbe9c10e550 2024-02-10T07:38:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fbe94003e90 con 0x7fbe9c10e550 2024-02-10T07:38:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.629+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fbe94004d30 con 0x7fbe9c10e550 2024-02-10T07:38:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.630+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe9c1148e0 con 0x7fbe9c10e550 2024-02-10T07:38:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.631+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fbe94003a00 con 0x7fbe9c10e550 2024-02-10T07:38:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.631+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fbe94087ad0 con 0x7fbe9c10e550 2024-02-10T07:38:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.631+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6801/3723357830 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe8407cd10 con 0x7fbe84079000 2024-02-10T07:38:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.631+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7fbe94087f00 con 0x7fbe9c10e550 2024-02-10T07:38:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.632+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== osd.0 v1:172.21.15.70:6801/3723357830 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fbe8407cd10 con 0x7fbe84079000 2024-02-10T07:38:44.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.648+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 --> v1:172.21.15.70:6801/3723357830 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe9c1a9780 con 0x7fbe84079000 2024-02-10T07:38:44.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbe997fa700 1 -- 172.21.15.70:0/116871625 <== osd.0 v1:172.21.15.70:6801/3723357830 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2546019811) 0x7fbe9c1a9780 con 0x7fbe84079000 2024-02-10T07:38:44.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 >> v1:172.21.15.70:6801/3723357830 conn(0x7fbe84079000 legacy=0x7fbe8407b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:44.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 >> v1:172.21.15.70:6800/4101821536 conn(0x7fbe8406e840 legacy=0x7fbe84070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:44.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 >> v1:172.21.15.70:6789/0 conn(0x7fbe9c10e550 legacy=0x7fbe9c1a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:44.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea0988700 1 -- 172.21.15.70:0/116871625 reap_dead start 2024-02-10T07:38:44.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 shutdown_connections 2024-02-10T07:38:44.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 >> 172.21.15.70:0/116871625 conn(0x7fbe9c0fd600 msgr2=0x7fbe9c101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:44.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 shutdown_connections 2024-02-10T07:38:44.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:44.649+0000 7fbea23eb700 1 -- 172.21.15.70:0/116871625 wait complete. 2024-02-10T07:38:45.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:44 smithi070 ceph-mon[39923]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:44 smithi070 ceph-mon[29548]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:44 smithi155 ceph-mon[32688]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:38:46.517 INFO:teuthology.orchestra.run.smithi070.stdout:167503724567 2024-02-10T07:38:46.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-10T07:38:47.027 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:46 smithi070 ceph-mon[39923]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-10T07:38:47.028 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:46 smithi070 ceph-mon[29548]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-10T07:38:47.050 INFO:teuthology.orchestra.run.smithi070.stdout:34359738416 2024-02-10T07:38:47.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-10T07:38:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:46 smithi155 ceph-mon[32688]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-10T07:38:47.478 INFO:teuthology.orchestra.run.smithi070.stdout:77309411366 2024-02-10T07:38:47.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-10T07:38:48.080 INFO:teuthology.orchestra.run.smithi070.stdout:201863462930 2024-02-10T07:38:48.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-10T07:38:48.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.682+0000 7fcddf1d6700 1 -- 172.21.15.70:0/4201164621 >> v1:172.21.15.70:6790/0 conn(0x7fcdd810cc00 legacy=0x7fcdd810f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:48.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.682+0000 7fcddf1d6700 1 -- 172.21.15.70:0/4201164621 shutdown_connections 2024-02-10T07:38:48.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.682+0000 7fcddf1d6700 1 -- 172.21.15.70:0/4201164621 >> 172.21.15.70:0/4201164621 conn(0x7fcdd8075cb0 msgr2=0x7fcdd80760c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:48.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.683+0000 7fcddf1d6700 1 -- 172.21.15.70:0/4201164621 shutdown_connections 2024-02-10T07:38:48.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.683+0000 7fcddf1d6700 1 -- 172.21.15.70:0/4201164621 wait complete. 2024-02-10T07:38:48.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.683+0000 7fcddf1d6700 1 Processor -- start 2024-02-10T07:38:48.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.683+0000 7fcddf1d6700 1 -- start start 2024-02-10T07:38:48.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcddf1d6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcdd81a69e0 con 0x7fcdd8109060 2024-02-10T07:38:48.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcddf1d6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcdd81a7b80 con 0x7fcdd8100550 2024-02-10T07:38:48.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcddf1d6700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcdd81a8d80 con 0x7fcdd810cc00 2024-02-10T07:38:48.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd7fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcdd8109060 0x7fcdd8104700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46944/0 (socket says 172.21.15.70:46944) 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd7fff700 1 -- 172.21.15.70:0/394005944 learned_addr learned my addr 172.21.15.70:0/394005944 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952174618 0 0) 0x7fcdd81a69e0 con 0x7fcdd8109060 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc8003480 con 0x7fcdd8109060 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144029184 0 0) 0x7fcdd81a8d80 con 0x7fcdd810cc00 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd81a69e0 con 0x7fcdd810cc00 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885897471 0 0) 0x7fcdd81a7b80 con 0x7fcdd8100550 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd81a8d80 con 0x7fcdd8100550 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2699616581 0 0) 0x7fcdc8003480 con 0x7fcdd8109060 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdd81a7b80 con 0x7fcdd8109060 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcdc40017f0 con 0x7fcdd8109060 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1680552184 0 0) 0x7fcdd81a69e0 con 0x7fcdd810cc00 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdc8003480 con 0x7fcdd810cc00 2024-02-10T07:38:48.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.684+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcdd0004660 con 0x7fcdd810cc00 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2712819444 0 0) 0x7fcdd81a8d80 con 0x7fcdd8100550 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdd81a69e0 con 0x7fcdd8100550 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcdcc0019f0 con 0x7fcdd8100550 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3702533978 0 0) 0x7fcdd81a7b80 con 0x7fcdd8109060 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 >> v1:172.21.15.70:6790/0 conn(0x7fcdd810cc00 legacy=0x7fcdd81a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 >> v1:172.21.15.155:6789/0 conn(0x7fcdd8100550 legacy=0x7fcdd8103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd81a9f80 con 0x7fcdd8109060 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd81a8f70 con 0x7fcdd8109060 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fcdc4001d60 con 0x7fcdd8109060 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcdc4005a80 con 0x7fcdd8109060 2024-02-10T07:38:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.685+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd81a93f0 con 0x7fcdd8109060 2024-02-10T07:38:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.686+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcdd81123b0 con 0x7fcdd8109060 2024-02-10T07:38:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.686+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fcdc401c240 con 0x7fcdd8109060 2024-02-10T07:38:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.687+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fcdc40888c0 con 0x7fcdd8109060 2024-02-10T07:38:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.687+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.155:6812/2127871237 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdc807cff0 con 0x7fcdc80792e0 2024-02-10T07:38:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.687+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7fcdc4088c80 con 0x7fcdd8109060 2024-02-10T07:38:48.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.688+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== osd.7 v1:172.21.15.155:6812/2127871237 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fcdc807cff0 con 0x7fcdc80792e0 2024-02-10T07:38:48.711 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.711+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 --> v1:172.21.15.155:6812/2127871237 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcdd8108290 con 0x7fcdc80792e0 2024-02-10T07:38:48.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.712+0000 7fcdd5ffb700 1 -- 172.21.15.70:0/394005944 <== osd.7 v1:172.21.15.155:6812/2127871237 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1312652062) 0x7fcdd8108290 con 0x7fcdc80792e0 2024-02-10T07:38:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.712+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 >> v1:172.21.15.155:6812/2127871237 conn(0x7fcdc80792e0 legacy=0x7fcdc807b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.712+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 >> v1:172.21.15.70:6800/4101821536 conn(0x7fcdc806eb00 legacy=0x7fcdc8070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.712+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 >> v1:172.21.15.70:6789/0 conn(0x7fcdd8109060 legacy=0x7fcdd8104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.713+0000 7fcddd773700 1 -- 172.21.15.70:0/394005944 reap_dead start 2024-02-10T07:38:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.713+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 shutdown_connections 2024-02-10T07:38:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.713+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 >> 172.21.15.70:0/394005944 conn(0x7fcdd8075cb0 msgr2=0x7fcdd8110100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.713+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 shutdown_connections 2024-02-10T07:38:48.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:48.713+0000 7fcddf1d6700 1 -- 172.21.15.70:0/394005944 wait complete. 2024-02-10T07:38:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:48 smithi070 ceph-mon[29548]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-10T07:38:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:48 smithi070 ceph-mon[39923]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-10T07:38:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:48 smithi155 ceph-mon[32688]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-10T07:38:50.055 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:49 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:50.056 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:49 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:49 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:38:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.639+0000 7fb7af567700 1 -- 172.21.15.70:0/2193467086 >> v1:172.21.15.70:6790/0 conn(0x7fb7a810e5f0 legacy=0x7fb7a8110a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.639+0000 7fb7af567700 1 -- 172.21.15.70:0/2193467086 shutdown_connections 2024-02-10T07:38:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.639+0000 7fb7af567700 1 -- 172.21.15.70:0/2193467086 >> 172.21.15.70:0/2193467086 conn(0x7fb7a80fc4c0 msgr2=0x7fb7a80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.639+0000 7fb7af567700 1 -- 172.21.15.70:0/2193467086 shutdown_connections 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.640+0000 7fb7af567700 1 -- 172.21.15.70:0/2193467086 wait complete. 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.640+0000 7fb7af567700 1 Processor -- start 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.640+0000 7fb7af567700 1 -- start start 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.640+0000 7fb7af567700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7a81a8de0 con 0x7fb7a810e5f0 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.640+0000 7fb7af567700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7a81a9fc0 con 0x7fb7a810aae0 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.640+0000 7fb7af567700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7a81ab1c0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.640+0000 7fb7adb04700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb7a810e5f0 0x7fb7a81a74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46950/0 (socket says 172.21.15.70:46950) 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb7ad303700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fb7a8102cd0 0x7fb7a810a3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:39346/0 (socket says 172.21.15.70:39346) 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb7adb04700 1 -- 172.21.15.70:0/3667040349 learned_addr learned my addr 172.21.15.70:0/3667040349 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840996567 0 0) 0x7fb7a81a8de0 con 0x7fb7a810e5f0 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7940035b0 con 0x7fb7a810e5f0 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335340400 0 0) 0x7fb7a81ab1c0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a81a8de0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77673031 0 0) 0x7fb7a81a9fc0 con 0x7fb7a810aae0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a81ab1c0 con 0x7fb7a810aae0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3897070395 0 0) 0x7fb7a81a8de0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7a81a9fc0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1482528592 0 0) 0x7fb7940035b0 con 0x7fb7a810e5f0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7a81a8de0 con 0x7fb7a810e5f0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb7a40018d0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb7a0001420 con 0x7fb7a810e5f0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3085069214 0 0) 0x7fb7a81ab1c0 con 0x7fb7a810aae0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7940035b0 con 0x7fb7a810aae0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb798001830 con 0x7fb7a810aae0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3547258313 0 0) 0x7fb7a81a9fc0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 >> v1:172.21.15.155:6789/0 conn(0x7fb7a810aae0 legacy=0x7fb7a81a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 >> v1:172.21.15.70:6789/0 conn(0x7fb7a810e5f0 legacy=0x7fb7a81a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a81ac3c0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a81ab3b0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.641+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a81ab830 con 0x7fb7a8102cd0 2024-02-10T07:38:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.642+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fb7a4003b30 con 0x7fb7a8102cd0 2024-02-10T07:38:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.642+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb7a4005b70 con 0x7fb7a8102cd0 2024-02-10T07:38:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.642+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb78c000f80 con 0x7fb7a8102cd0 2024-02-10T07:38:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.643+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fb7a40036a0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.643+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fb7a40888c0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.644+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6813/3512826399 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb79407cde0 con 0x7fb7940790d0 2024-02-10T07:38:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.644+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7fb7a4088ce0 con 0x7fb7a8102cd0 2024-02-10T07:38:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.644+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== osd.3 v1:172.21.15.70:6813/3512826399 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fb79407cde0 con 0x7fb7940790d0 2024-02-10T07:38:50.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.660+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 --> v1:172.21.15.70:6813/3512826399 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb78c002c40 con 0x7fb7940790d0 2024-02-10T07:38:50.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.661+0000 7fb79e7fc700 1 -- 172.21.15.70:0/3667040349 <== osd.3 v1:172.21.15.70:6813/3512826399 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1221308824) 0x7fb78c002c40 con 0x7fb7940790d0 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.661+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 >> v1:172.21.15.70:6813/3512826399 conn(0x7fb7940790d0 legacy=0x7fb79407b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.661+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 >> v1:172.21.15.70:6800/4101821536 conn(0x7fb79406e9f0 legacy=0x7fb794070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.661+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 >> v1:172.21.15.70:6790/0 conn(0x7fb7a8102cd0 legacy=0x7fb7a810a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.661+0000 7fb7adb04700 1 -- 172.21.15.70:0/3667040349 reap_dead start 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.661+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 shutdown_connections 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.661+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 >> 172.21.15.70:0/3667040349 conn(0x7fb7a80fc4c0 msgr2=0x7fb7a80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.662+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 shutdown_connections 2024-02-10T07:38:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.662+0000 7fb7af567700 1 -- 172.21.15.70:0/3667040349 wait complete. 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.856+0000 7f1c3cc72700 1 -- 172.21.15.70:0/190184604 >> v1:172.21.15.70:6790/0 conn(0x7f1c3810a990 legacy=0x7f1c3810cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.856+0000 7f1c3cc72700 1 -- 172.21.15.70:0/190184604 shutdown_connections 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.856+0000 7f1c3cc72700 1 -- 172.21.15.70:0/190184604 >> 172.21.15.70:0/190184604 conn(0x7f1c380fd5f0 msgr2=0x7f1c380ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.856+0000 7f1c3cc72700 1 -- 172.21.15.70:0/190184604 shutdown_connections 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.856+0000 7f1c3cc72700 1 -- 172.21.15.70:0/190184604 wait complete. 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.857+0000 7f1c3cc72700 1 Processor -- start 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.857+0000 7f1c3cc72700 1 -- start start 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.857+0000 7f1c3cc72700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1c381a6980 con 0x7f1c3810a990 2024-02-10T07:38:50.879 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.857+0000 7f1c3cc72700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1c381a7b60 con 0x7f1c38107b70 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.857+0000 7f1c3cc72700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1c381a8d60 con 0x7f1c3810e560 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.857+0000 7f1c35d9b700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1c3810a990 0x7f1c381a1910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46970/0 (socket says 172.21.15.70:46970) 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.857+0000 7f1c35d9b700 1 -- 172.21.15.70:0/2633565467 learned_addr learned my addr 172.21.15.70:0/2633565467 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444037823 0 0) 0x7f1c381a6980 con 0x7f1c3810a990 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c200035b0 con 0x7f1c3810a990 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375836837 0 0) 0x7f1c381a8d60 con 0x7f1c3810e560 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c381a6980 con 0x7f1c3810e560 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582398458 0 0) 0x7f1c381a7b60 con 0x7f1c38107b70 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c381a8d60 con 0x7f1c38107b70 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3171409495 0 0) 0x7f1c200035b0 con 0x7f1c3810a990 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1c381a7b60 con 0x7f1c3810a990 2024-02-10T07:38:50.880 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1c2c004440 con 0x7f1c3810a990 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2556108769 0 0) 0x7f1c381a6980 con 0x7f1c3810e560 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1c200035b0 con 0x7f1c3810e560 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1c30001b70 con 0x7f1c3810e560 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 588225368 0 0) 0x7f1c381a8d60 con 0x7f1c38107b70 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1c381a6980 con 0x7f1c38107b70 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1c28001af0 con 0x7f1c38107b70 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1229301837 0 0) 0x7f1c381a7b60 con 0x7f1c3810a990 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 >> v1:172.21.15.70:6790/0 conn(0x7f1c3810e560 legacy=0x7f1c381a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 >> v1:172.21.15.155:6789/0 conn(0x7f1c38107b70 legacy=0x7f1c3810dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c381a9f60 con 0x7f1c3810a990 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c381a8f50 con 0x7f1c3810a990 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c381a93d0 con 0x7f1c3810a990 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f1c2c001080 con 0x7f1c3810a990 2024-02-10T07:38:50.881 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.858+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f1c2c004ee0 con 0x7f1c3810a990 2024-02-10T07:38:50.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.859+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c18000fb0 con 0x7f1c3810a990 2024-02-10T07:38:50.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.860+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f1c2c0050e0 con 0x7f1c3810a990 2024-02-10T07:38:50.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.860+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f1c2c088a10 con 0x7f1c3810a990 2024-02-10T07:38:50.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.860+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.155:6800/1901321685 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c2007cf00 con 0x7f1c200791f0 2024-02-10T07:38:50.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.860+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f1c2c088e70 con 0x7f1c3810a990 2024-02-10T07:38:50.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.862+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== osd.4 v1:172.21.15.155:6800/1901321685 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f1c2007cf00 con 0x7f1c200791f0 2024-02-10T07:38:50.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.878+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 --> v1:172.21.15.155:6800/1901321685 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c18002c70 con 0x7f1c200791f0 2024-02-10T07:38:50.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.879+0000 7f1c277fe700 1 -- 172.21.15.70:0/2633565467 <== osd.4 v1:172.21.15.155:6800/1901321685 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3087939264) 0x7f1c18002c70 con 0x7f1c200791f0 2024-02-10T07:38:50.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.879+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 >> v1:172.21.15.155:6800/1901321685 conn(0x7f1c200791f0 legacy=0x7f1c2007b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.879+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 >> v1:172.21.15.70:6800/4101821536 conn(0x7f1c2006ea80 legacy=0x7f1c20070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.879+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 >> v1:172.21.15.70:6789/0 conn(0x7f1c3810a990 legacy=0x7f1c381a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:50.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.879+0000 7f1c36d9d700 1 -- 172.21.15.70:0/2633565467 reap_dead start 2024-02-10T07:38:50.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.880+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 shutdown_connections 2024-02-10T07:38:50.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.880+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 >> 172.21.15.70:0/2633565467 conn(0x7f1c380fd5f0 msgr2=0x7f1c38111a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:50.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.880+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 shutdown_connections 2024-02-10T07:38:50.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:50.880+0000 7f1c3cc72700 1 -- 172.21.15.70:0/2633565467 wait complete. 2024-02-10T07:38:51.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:50 smithi070 ceph-mon[39923]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-10T07:38:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:50 smithi070 ceph-mon[29548]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-10T07:38:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:50 smithi155 ceph-mon[32688]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-10T07:38:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.284+0000 7f0891358700 1 -- 172.21.15.70:0/678318596 >> v1:172.21.15.70:6789/0 conn(0x7f088c10a9f0 legacy=0x7f088c10cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.284+0000 7f0891358700 1 -- 172.21.15.70:0/678318596 shutdown_connections 2024-02-10T07:38:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.284+0000 7f0891358700 1 -- 172.21.15.70:0/678318596 >> 172.21.15.70:0/678318596 conn(0x7f088c075ca0 msgr2=0x7f088c0760b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.285+0000 7f0891358700 1 -- 172.21.15.70:0/678318596 shutdown_connections 2024-02-10T07:38:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.285+0000 7f0891358700 1 -- 172.21.15.70:0/678318596 wait complete. 2024-02-10T07:38:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.285+0000 7f0891358700 1 Processor -- start 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.285+0000 7f0891358700 1 -- start start 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0891358700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f088c10d910 con 0x7f088c100570 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0891358700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f088c1a9d40 con 0x7f088c10e560 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0891358700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f088c1aaee0 con 0x7f088c10a9f0 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f088affd700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f088c100570 0x7f088c10cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46986/0 (socket says 172.21.15.70:46986) 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f088affd700 1 -- 172.21.15.70:0/1081120513 learned_addr learned my addr 172.21.15.70:0/1081120513 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292352658 0 0) 0x7f088c10d910 con 0x7f088c100570 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08740035b0 con 0x7f088c100570 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805360420 0 0) 0x7f088c1aaee0 con 0x7f088c10a9f0 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f088c10d910 con 0x7f088c10a9f0 2024-02-10T07:38:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838775158 0 0) 0x7f088c1a9d40 con 0x7f088c10e560 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f088c1aaee0 con 0x7f088c10e560 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2284893788 0 0) 0x7f08740035b0 con 0x7f088c100570 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f088c1a9d40 con 0x7f088c100570 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f087c001950 con 0x7f088c100570 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3805874987 0 0) 0x7f088c10d910 con 0x7f088c10a9f0 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f08740035b0 con 0x7f088c10a9f0 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0880004540 con 0x7f088c10a9f0 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3630538684 0 0) 0x7f088c1aaee0 con 0x7f088c10e560 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f088c10d910 con 0x7f088c10e560 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0884001c30 con 0x7f088c10e560 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.286+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 729539583 0 0) 0x7f088c1a9d40 con 0x7f088c100570 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 >> v1:172.21.15.70:6790/0 conn(0x7f088c10a9f0 legacy=0x7f088c1a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 >> v1:172.21.15.155:6789/0 conn(0x7f088c10e560 legacy=0x7f088c1a7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c1ac0e0 con 0x7f088c100570 2024-02-10T07:38:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c1ab0d0 con 0x7f088c100570 2024-02-10T07:38:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c1ab5d0 con 0x7f088c100570 2024-02-10T07:38:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f087c003b10 con 0x7f088c100570 2024-02-10T07:38:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f087c005b00 con 0x7f088c100570 2024-02-10T07:38:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.287+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0878000f80 con 0x7f088c100570 2024-02-10T07:38:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.288+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f087c01c190 con 0x7f088c100570 2024-02-10T07:38:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.289+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f087c088750 con 0x7f088c100570 2024-02-10T07:38:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.289+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6805/918939980 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f087407cf20 con 0x7f0874079210 2024-02-10T07:38:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.289+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f087c088be0 con 0x7f088c100570 2024-02-10T07:38:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.290+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== osd.1 v1:172.21.15.70:6805/918939980 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f087407cf20 con 0x7f0874079210 2024-02-10T07:38:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.306+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 --> v1:172.21.15.70:6805/918939980 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0878002c40 con 0x7f0874079210 2024-02-10T07:38:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.306+0000 7f0873fff700 1 -- 172.21.15.70:0/1081120513 <== osd.1 v1:172.21.15.70:6805/918939980 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3910016750) 0x7f0878002c40 con 0x7f0874079210 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 >> v1:172.21.15.70:6805/918939980 conn(0x7f0874079210 legacy=0x7f087407b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 >> v1:172.21.15.70:6800/4101821536 conn(0x7f087406eaa0 legacy=0x7f0874070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 >> v1:172.21.15.70:6789/0 conn(0x7f088c100570 legacy=0x7f088c10cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f088b7fe700 1 -- 172.21.15.70:0/1081120513 reap_dead start 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 shutdown_connections 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 >> 172.21.15.70:0/1081120513 conn(0x7f088c075ca0 msgr2=0x7f088c109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 shutdown_connections 2024-02-10T07:38:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:38:51.307+0000 7f0891358700 1 -- 172.21.15.70:0/1081120513 wait complete. 2024-02-10T07:38:51.719 INFO:teuthology.orchestra.run.smithi070.stdout:133143986206 2024-02-10T07:38:51.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-10T07:38:52.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:38:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:38:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:38:52.365 INFO:teuthology.orchestra.run.smithi070.stdout:111669149730 2024-02-10T07:38:52.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-10T07:38:53.119 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:52 smithi070 ceph-mon[29548]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:53.119 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:52 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:38:53.119 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:52 smithi070 ceph-mon[39923]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:53.119 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:52 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:38:53.144 INFO:teuthology.orchestra.run.smithi070.stdout:55834574891 2024-02-10T07:38:53.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-10T07:38:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:52 smithi155 ceph-mon[32688]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:52 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:38:53.541 INFO:teuthology.orchestra.run.smithi070.stdout:236223201294 2024-02-10T07:38:53.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-10T07:38:53.608 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:53.616 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:53.617 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:53.619 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:53.631 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:53.635 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:55.202 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:54 smithi070 ceph-mon[29548]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:55.202 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:54 smithi070 ceph-mon[39923]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:54 smithi155 ceph-mon[32688]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:55.548 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:38:57.145 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:56 smithi070 ceph-mon[39923]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:57.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:56 smithi070 ceph-mon[29548]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:56 smithi155 ceph-mon[32688]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:38:59.155 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:58 smithi070 ceph-mon[29548]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:38:59.156 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:58 smithi070 ceph-mon[39923]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:38:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:58 smithi155 ceph-mon[32688]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:00.100 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:38:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:00.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:38:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:38:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:01.139 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:00 smithi070 ceph-mon[29548]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:01.140 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:00 smithi070 ceph-mon[39923]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.156+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4193268870 >> v1:172.21.15.70:6789/0 conn(0x7fdc34109860 legacy=0x7fdc3410bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.156+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4193268870 shutdown_connections 2024-02-10T07:39:01.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.156+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4193268870 >> 172.21.15.70:0/4193268870 conn(0x7fdc340fc4c0 msgr2=0x7fdc340fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.156+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4193268870 shutdown_connections 2024-02-10T07:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.156+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4193268870 wait complete. 2024-02-10T07:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc3aca6700 1 Processor -- start 2024-02-10T07:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc3aca6700 1 -- start start 2024-02-10T07:39:01.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc3aca6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdc341a5800 con 0x7fdc3410d430 2024-02-10T07:39:01.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc3aca6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdc341a6a00 con 0x7fdc341069b0 2024-02-10T07:39:01.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc3aca6700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdc341a7c00 con 0x7fdc34109860 2024-02-10T07:39:01.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc33fff700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fdc34109860 0x7fdc341a07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:43426/0 (socket says 172.21.15.70:43426) 2024-02-10T07:39:01.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc39243700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdc3410d430 0x7fdc341a3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32858/0 (socket says 172.21.15.70:32858) 2024-02-10T07:39:01.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc33fff700 1 -- 172.21.15.70:0/4009304498 learned_addr learned my addr 172.21.15.70:0/4009304498 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:01.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.157+0000 7fdc38a42700 1 --1- 172.21.15.70:0/4009304498 >> v1:172.21.15.155:6789/0 conn(0x7fdc341069b0 0x7fdc3410ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:52898/0 (socket says 172.21.15.70:52898) 2024-02-10T07:39:01.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238257585 0 0) 0x7fdc341a7c00 con 0x7fdc34109860 2024-02-10T07:39:01.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc240035b0 con 0x7fdc34109860 2024-02-10T07:39:01.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165502856 0 0) 0x7fdc341a5800 con 0x7fdc3410d430 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc341a7c00 con 0x7fdc3410d430 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443001323 0 0) 0x7fdc341a6a00 con 0x7fdc341069b0 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc341a5800 con 0x7fdc341069b0 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1653316373 0 0) 0x7fdc240035b0 con 0x7fdc34109860 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc341a6a00 con 0x7fdc34109860 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fdc20004540 con 0x7fdc34109860 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1584674198 0 0) 0x7fdc341a5800 con 0x7fdc341069b0 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc240035b0 con 0x7fdc341069b0 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 299055209 0 0) 0x7fdc341a6a00 con 0x7fdc34109860 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 >> v1:172.21.15.155:6789/0 conn(0x7fdc341069b0 legacy=0x7fdc3410ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 >> v1:172.21.15.70:6789/0 conn(0x7fdc3410d430 legacy=0x7fdc341a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc341a8e00 con 0x7fdc34109860 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc341a59f0 con 0x7fdc34109860 2024-02-10T07:39:01.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fdc20003c10 con 0x7fdc34109860 2024-02-10T07:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.158+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fdc20004f10 con 0x7fdc34109860 2024-02-10T07:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.159+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc341a5e70 con 0x7fdc34109860 2024-02-10T07:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.160+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc341020f0 con 0x7fdc34109860 2024-02-10T07:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.160+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fdc20005090 con 0x7fdc34109860 2024-02-10T07:39:01.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.160+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fdc20088d80 con 0x7fdc34109860 2024-02-10T07:39:01.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.165+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fdc200b8d20 con 0x7fdc34109860 2024-02-10T07:39:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:00 smithi155 ceph-mon[32688]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:01.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.374+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdc3410b060 con 0x7fdc34109860 2024-02-10T07:39:01.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.374+0000 7fdc31ffb700 1 -- 172.21.15.70:0/4009304498 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 4158404283) 0x7fdc20056020 con 0x7fdc34109860 2024-02-10T07:39:01.375 INFO:teuthology.orchestra.run.smithi070.stdout:77309411370 2024-02-10T07:39:01.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.376+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 >> v1:172.21.15.70:6800/4101821536 conn(0x7fdc2406e730 legacy=0x7fdc24070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.376+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 >> v1:172.21.15.70:6790/0 conn(0x7fdc34109860 legacy=0x7fdc341a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.376+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 shutdown_connections 2024-02-10T07:39:01.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.376+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 >> 172.21.15.70:0/4009304498 conn(0x7fdc340fc4c0 msgr2=0x7fdc341100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:01.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.377+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 shutdown_connections 2024-02-10T07:39:01.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.377+0000 7fdc3aca6700 1 -- 172.21.15.70:0/4009304498 wait complete. 2024-02-10T07:39:01.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.626+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/981886718 >> v1:172.21.15.70:6789/0 conn(0x7f4e4810e550 legacy=0x7f4e48110a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.626+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/981886718 shutdown_connections 2024-02-10T07:39:01.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.626+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/981886718 >> 172.21.15.70:0/981886718 conn(0x7f4e480fd600 msgr2=0x7f4e480ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:01.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.626+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/981886718 shutdown_connections 2024-02-10T07:39:01.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.626+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/981886718 wait complete. 2024-02-10T07:39:01.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.627+0000 7f4e4e0f6700 1 Processor -- start 2024-02-10T07:39:01.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.627+0000 7f4e4e0f6700 1 -- start start 2024-02-10T07:39:01.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.627+0000 7f4e4e0f6700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e481a8b40 con 0x7f4e48107b60 2024-02-10T07:39:01.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.627+0000 7f4e4e0f6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e481a9ce0 con 0x7f4e4810a980 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.627+0000 7f4e4e0f6700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e481aae80 con 0x7f4e4810e550 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.627+0000 7f4e477fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4e48107b60 0x7f4e48075800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32886/0 (socket says 172.21.15.70:32886) 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.627+0000 7f4e477fe700 1 -- 172.21.15.70:0/8500232 learned_addr learned my addr 172.21.15.70:0/8500232 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098353684 0 0) 0x7f4e481aae80 con 0x7f4e4810e550 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e300049e0 con 0x7f4e4810e550 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490007143 0 0) 0x7f4e481a8b40 con 0x7f4e48107b60 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e481aae80 con 0x7f4e48107b60 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856852231 0 0) 0x7f4e481a9ce0 con 0x7f4e4810a980 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e481a8b40 con 0x7f4e4810a980 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3720873597 0 0) 0x7f4e300049e0 con 0x7f4e4810e550 2024-02-10T07:39:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e481a9ce0 con 0x7f4e4810e550 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4e40004800 con 0x7f4e4810e550 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3694324004 0 0) 0x7f4e481aae80 con 0x7f4e48107b60 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e300049e0 con 0x7f4e48107b60 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4e380017f0 con 0x7f4e48107b60 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1567609245 0 0) 0x7f4e481a8b40 con 0x7f4e4810a980 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e481aae80 con 0x7f4e4810a980 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4e3c001850 con 0x7f4e4810a980 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2786351164 0 0) 0x7f4e481a9ce0 con 0x7f4e4810e550 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 >> v1:172.21.15.155:6789/0 conn(0x7f4e4810a980 legacy=0x7f4e481a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 >> v1:172.21.15.70:6789/0 conn(0x7f4e48107b60 legacy=0x7f4e48075800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e481ac060 con 0x7f4e4810e550 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e481a8d30 con 0x7f4e4810e550 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e481a91b0 con 0x7f4e4810e550 2024-02-10T07:39:01.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f4e40001e00 con 0x7f4e4810e550 2024-02-10T07:39:01.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.628+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4e40005080 con 0x7f4e4810e550 2024-02-10T07:39:01.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.629+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e34005150 con 0x7f4e4810e550 2024-02-10T07:39:01.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.629+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f4e4001b710 con 0x7f4e4810e550 2024-02-10T07:39:01.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.630+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f4e40087d00 con 0x7f4e4810e550 2024-02-10T07:39:01.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.635+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f4e40054fe0 con 0x7f4e4810e550 2024-02-10T07:39:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.836+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4e34005400 con 0x7f4e4810e550 2024-02-10T07:39:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.836+0000 7f4e44ff9700 1 -- 172.21.15.70:0/8500232 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (unknown 1823589948 0 1544327744) 0x7f4e40053350 con 0x7f4e4810e550 2024-02-10T07:39:01.836 INFO:teuthology.orchestra.run.smithi070.stdout:133143986208 2024-02-10T07:39:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.838+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 >> v1:172.21.15.70:6800/4101821536 conn(0x7f4e3006ebd0 legacy=0x7f4e30071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.838+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 >> v1:172.21.15.70:6790/0 conn(0x7f4e4810e550 legacy=0x7f4e481a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.838+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 shutdown_connections 2024-02-10T07:39:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.838+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 >> 172.21.15.70:0/8500232 conn(0x7f4e480fd600 msgr2=0x7f4e48111210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.838+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 shutdown_connections 2024-02-10T07:39:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:01.838+0000 7f4e4e0f6700 1 -- 172.21.15.70:0/8500232 wait complete. 2024-02-10T07:39:02.149 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4009304498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T07:39:02.149 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/8500232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T07:39:02.150 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4009304498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T07:39:02.150 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/8500232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T07:39:02.150 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:39:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:39:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4009304498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T07:39:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/8500232' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T07:39:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.335+0000 7f95ed7bd700 1 -- 172.21.15.70:0/1708954226 >> v1:172.21.15.70:6789/0 conn(0x7f95e810e590 legacy=0x7f95e8110a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.335+0000 7f95ed7bd700 1 -- 172.21.15.70:0/1708954226 shutdown_connections 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.335+0000 7f95ed7bd700 1 -- 172.21.15.70:0/1708954226 >> 172.21.15.70:0/1708954226 conn(0x7f95e80fc4c0 msgr2=0x7f95e80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.335+0000 7f95ed7bd700 1 -- 172.21.15.70:0/1708954226 shutdown_connections 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.335+0000 7f95ed7bd700 1 -- 172.21.15.70:0/1708954226 wait complete. 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.335+0000 7f95ed7bd700 1 Processor -- start 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95ed7bd700 1 -- start start 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95ed7bd700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95e81a6ac0 con 0x7f95e810e590 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95ed7bd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95e81a7ca0 con 0x7f95e810aa80 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95ed7bd700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95e81a8ea0 con 0x7f95e8102c70 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e6ffd700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f95e8102c70 0x7f95e810a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:43456/0 (socket says 172.21.15.70:43456) 2024-02-10T07:39:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e6ffd700 1 -- 172.21.15.70:0/470758054 learned_addr learned my addr 172.21.15.70:0/470758054 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e77fe700 1 --1- 172.21.15.70:0/470758054 >> v1:172.21.15.70:6789/0 conn(0x7f95e810e590 0x7f95e81a5180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32912/0 (socket says 172.21.15.70:32912) 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425217239 0 0) 0x7f95e81a6ac0 con 0x7f95e810e590 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d40031f0 con 0x7f95e810e590 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388514582 0 0) 0x7f95e81a8ea0 con 0x7f95e8102c70 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e81a6ac0 con 0x7f95e8102c70 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611894938 0 0) 0x7f95e81a7ca0 con 0x7f95e810aa80 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e81a8ea0 con 0x7f95e810aa80 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3884476780 0 0) 0x7f95e81a6ac0 con 0x7f95e8102c70 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95e81a7ca0 con 0x7f95e8102c70 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f95d80019a0 con 0x7f95e8102c70 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2864983904 0 0) 0x7f95d40031f0 con 0x7f95e810e590 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95e81a6ac0 con 0x7f95e810e590 2024-02-10T07:39:02.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.336+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f95e00017b0 con 0x7f95e810e590 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3189667214 0 0) 0x7f95e81a8ea0 con 0x7f95e810aa80 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95d40031f0 con 0x7f95e810aa80 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f95d0001830 con 0x7f95e810aa80 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3658600364 0 0) 0x7f95e81a7ca0 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 >> v1:172.21.15.155:6789/0 conn(0x7f95e810aa80 legacy=0x7f95e81a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 >> v1:172.21.15.70:6789/0 conn(0x7f95e810e590 legacy=0x7f95e81a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e81aa0a0 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e81a9090 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e81a9510 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f95d8001c40 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.337+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f95d8005c50 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.338+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c8005120 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.338+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f95d8003930 con 0x7f95e8102c70 2024-02-10T07:39:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.339+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f95d8088830 con 0x7f95e8102c70 2024-02-10T07:39:02.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.343+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f95d8055b30 con 0x7f95e8102c70 2024-02-10T07:39:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.543+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f95c80053d0 con 0x7f95e8102c70 2024-02-10T07:39:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.544+0000 7f95e4ff9700 1 -- 172.21.15.70:0/470758054 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 2087499782) 0x7f95d8053ea0 con 0x7f95e8102c70 2024-02-10T07:39:02.544 INFO:teuthology.orchestra.run.smithi070.stdout:34359738419 2024-02-10T07:39:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.546+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 >> v1:172.21.15.70:6800/4101821536 conn(0x7f95d406ec60 legacy=0x7f95d4071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.546+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 >> v1:172.21.15.70:6790/0 conn(0x7f95e8102c70 legacy=0x7f95e810a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.546+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 shutdown_connections 2024-02-10T07:39:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.546+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 >> 172.21.15.70:0/470758054 conn(0x7f95e80fc4c0 msgr2=0x7f95e81109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:02.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.546+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 shutdown_connections 2024-02-10T07:39:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:02.546+0000 7f95ed7bd700 1 -- 172.21.15.70:0/470758054 wait complete. 2024-02-10T07:39:02.965 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411370 for osd.2 2024-02-10T07:39:02.965 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:03.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.146+0000 7f0fc4718700 1 -- 172.21.15.70:0/2933754203 >> v1:172.21.15.70:6789/0 conn(0x7f0fbc10d430 legacy=0x7f0fbc10f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:03.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.146+0000 7f0fc4718700 1 -- 172.21.15.70:0/2933754203 shutdown_connections 2024-02-10T07:39:03.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.146+0000 7f0fc4718700 1 -- 172.21.15.70:0/2933754203 >> 172.21.15.70:0/2933754203 conn(0x7f0fbc0fc4c0 msgr2=0x7f0fbc0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:03.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.146+0000 7f0fc4718700 1 -- 172.21.15.70:0/2933754203 shutdown_connections 2024-02-10T07:39:03.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.146+0000 7f0fc4718700 1 -- 172.21.15.70:0/2933754203 wait complete. 2024-02-10T07:39:03.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.147+0000 7f0fc4718700 1 Processor -- start 2024-02-10T07:39:03.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.148+0000 7f0fc4718700 1 -- start start 2024-02-10T07:39:03.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.148+0000 7f0fc24b4700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f0fbc1069b0 0x7f0fbc10caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:43470/0 (socket says 172.21.15.70:43470) 2024-02-10T07:39:03.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.148+0000 7f0fc24b4700 1 -- 172.21.15.70:0/3290013573 learned_addr learned my addr 172.21.15.70:0/3290013573 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:03.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.148+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0fbc1a5890 con 0x7f0fbc10d430 2024-02-10T07:39:03.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0fbc1a6a70 con 0x7f0fbc109860 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0fbc1a5a60 con 0x7f0fbc1069b0 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739728920 0 0) 0x7f0fbc1a5890 con 0x7f0fbc10d430 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fa80035b0 con 0x7f0fbc10d430 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465827602 0 0) 0x7f0fbc1a6a70 con 0x7f0fbc109860 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fbc1a5890 con 0x7f0fbc109860 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940646319 0 0) 0x7f0fbc1a5a60 con 0x7f0fbc1069b0 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fbc1a6a70 con 0x7f0fbc1069b0 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3598743960 0 0) 0x7f0fbc1a6a70 con 0x7f0fbc1069b0 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fbc1a5a60 con 0x7f0fbc1069b0 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0fb80019f0 con 0x7f0fbc1069b0 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3737768015 0 0) 0x7f0fbc1a5890 con 0x7f0fbc109860 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fbc1a6a70 con 0x7f0fbc109860 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3639624441 0 0) 0x7f0fa80035b0 con 0x7f0fbc10d430 2024-02-10T07:39:03.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.149+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fbc1a5890 con 0x7f0fbc10d430 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0fac0017b0 con 0x7f0fbc109860 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0fb40046f0 con 0x7f0fbc10d430 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1861387357 0 0) 0x7f0fbc1a5a60 con 0x7f0fbc1069b0 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 >> v1:172.21.15.155:6789/0 conn(0x7f0fbc109860 legacy=0x7f0fbc1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 >> v1:172.21.15.70:6789/0 conn(0x7f0fbc10d430 legacy=0x7f0fbc1a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc1a5c30 con 0x7f0fbc1069b0 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc1a5ee0 con 0x7f0fbc1069b0 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc1a6360 con 0x7f0fbc1069b0 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f0fb8003b10 con 0x7f0fbc1069b0 2024-02-10T07:39:03.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.150+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f0fb8005bf0 con 0x7f0fbc1069b0 2024-02-10T07:39:03.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.151+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa0005120 con 0x7f0fbc1069b0 2024-02-10T07:39:03.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.152+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f0fb8003680 con 0x7f0fbc1069b0 2024-02-10T07:39:03.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.152+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f0fb8088770 con 0x7f0fbc1069b0 2024-02-10T07:39:03.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.157+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f0fb8055a70 con 0x7f0fbc1069b0 2024-02-10T07:39:03.235 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:02 smithi070 ceph-mon[39923]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:03.235 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/470758054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T07:39:03.235 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:02 smithi070 ceph-mon[29548]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:03.236 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/470758054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T07:39:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:02 smithi155 ceph-mon[32688]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/470758054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T07:39:03.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.359+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0fa00053d0 con 0x7f0fbc1069b0 2024-02-10T07:39:03.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.359+0000 7f0fb37fe700 1 -- 172.21.15.70:0/3290013573 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (unknown 1274345170 0 3537426591) 0x7f0fb8053de0 con 0x7f0fbc1069b0 2024-02-10T07:39:03.360 INFO:teuthology.orchestra.run.smithi070.stdout:201863462934 2024-02-10T07:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.361+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 >> v1:172.21.15.70:6800/4101821536 conn(0x7f0fa8073070 legacy=0x7f0fa8075530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.361+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 >> v1:172.21.15.70:6790/0 conn(0x7f0fbc1069b0 legacy=0x7f0fbc10caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.361+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 shutdown_connections 2024-02-10T07:39:03.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.361+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 >> 172.21.15.70:0/3290013573 conn(0x7f0fbc0fc4c0 msgr2=0x7f0fbc0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:03.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.361+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 shutdown_connections 2024-02-10T07:39:03.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:03.361+0000 7f0fc4718700 1 -- 172.21.15.70:0/3290013573 wait complete. 2024-02-10T07:39:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3290013573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T07:39:04.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3290013573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T07:39:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3290013573' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T07:39:04.760 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986206 got 133143986208 for osd.4 2024-02-10T07:39:04.761 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:04.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:04 smithi070 ceph-mon[29548]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:04.979 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:04 smithi070 ceph-mon[39923]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:04 smithi155 ceph-mon[32688]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:05.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.509+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/4175548214 >> v1:172.21.15.70:6789/0 conn(0x7f7eb81069b0 legacy=0x7f7eb8106d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:05.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.509+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/4175548214 shutdown_connections 2024-02-10T07:39:05.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.509+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/4175548214 >> 172.21.15.70:0/4175548214 conn(0x7f7eb80fc4c0 msgr2=0x7f7eb80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:05.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.509+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/4175548214 shutdown_connections 2024-02-10T07:39:05.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.509+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/4175548214 wait complete. 2024-02-10T07:39:05.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7ebfcdb700 1 Processor -- start 2024-02-10T07:39:05.510 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7ebfcdb700 1 -- start start 2024-02-10T07:39:05.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7ebfcdb700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7eb81a6a00 con 0x7f7eb81069b0 2024-02-10T07:39:05.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7ebfcdb700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7eb81a7be0 con 0x7f7eb810d430 2024-02-10T07:39:05.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7ebfcdb700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7eb81a8de0 con 0x7f7eb8109860 2024-02-10T07:39:05.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7ebd276700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f7eb8109860 0x7f7eb81a17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:43482/0 (socket says 172.21.15.70:43482) 2024-02-10T07:39:05.511 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7ebd276700 1 -- 172.21.15.70:0/3297510492 learned_addr learned my addr 172.21.15.70:0/3297510492 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532511033 0 0) 0x7f7eb81a8de0 con 0x7f7eb8109860 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea40031f0 con 0x7f7eb8109860 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625979666 0 0) 0x7f7eb81a6a00 con 0x7f7eb81069b0 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb81a8de0 con 0x7f7eb81069b0 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566198937 0 0) 0x7f7eb81a7be0 con 0x7f7eb810d430 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb81a6a00 con 0x7f7eb810d430 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1052081700 0 0) 0x7f7ea40031f0 con 0x7f7eb8109860 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7eb81a7be0 con 0x7f7eb8109860 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7ea80018f0 con 0x7f7eb8109860 2024-02-10T07:39:05.513 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2803622546 0 0) 0x7f7eb81a8de0 con 0x7f7eb81069b0 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.510+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ea40031f0 con 0x7f7eb81069b0 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7eb4001960 con 0x7f7eb81069b0 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3625433718 0 0) 0x7f7eb81a6a00 con 0x7f7eb810d430 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7eb81a8de0 con 0x7f7eb810d430 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7eb00013f0 con 0x7f7eb810d430 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 594127851 0 0) 0x7f7eb81a7be0 con 0x7f7eb8109860 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 >> v1:172.21.15.155:6789/0 conn(0x7f7eb810d430 legacy=0x7f7eb81a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 >> v1:172.21.15.70:6789/0 conn(0x7f7eb81069b0 legacy=0x7f7eb810cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb81a9fe0 con 0x7f7eb8109860 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb81a6bf0 con 0x7f7eb8109860 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb81a7070 con 0x7f7eb8109860 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f7ea8003b30 con 0x7f7eb8109860 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.511+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7ea8005bd0 con 0x7f7eb8109860 2024-02-10T07:39:05.514 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.512+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c005120 con 0x7f7eb8109860 2024-02-10T07:39:05.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.512+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f7ea801c340 con 0x7f7eb8109860 2024-02-10T07:39:05.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.513+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f7ea8088a70 con 0x7f7eb8109860 2024-02-10T07:39:05.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.517+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f7ea8055ac0 con 0x7f7eb8109860 2024-02-10T07:39:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.713+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7e9c0053d0 con 0x7f7eb8109860 2024-02-10T07:39:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.714+0000 7f7eaeffd700 1 -- 172.21.15.70:0/3297510492 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 50725365) 0x7f7ea80540c0 con 0x7f7eb8109860 2024-02-10T07:39:05.714 INFO:teuthology.orchestra.run.smithi070.stdout:55834574893 2024-02-10T07:39:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.716+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 >> v1:172.21.15.70:6800/4101821536 conn(0x7f7ea406ea70 legacy=0x7f7ea4070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.716+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 >> v1:172.21.15.70:6790/0 conn(0x7f7eb8109860 legacy=0x7f7eb81a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.716+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 shutdown_connections 2024-02-10T07:39:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.716+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 >> 172.21.15.70:0/3297510492 conn(0x7f7eb80fc4c0 msgr2=0x7f7eb806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.716+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 shutdown_connections 2024-02-10T07:39:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:05.716+0000 7f7ebfcdb700 1 -- 172.21.15.70:0/3297510492 wait complete. 2024-02-10T07:39:06.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738416 got 34359738419 for osd.0 2024-02-10T07:39:06.087 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3297510492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T07:39:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3297510492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T07:39:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3297510492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T07:39:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.281+0000 7fb350496700 1 -- 172.21.15.70:0/1747654883 >> v1:172.21.15.70:6789/0 conn(0x7fb34810d430 legacy=0x7fb34810f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.281+0000 7fb350496700 1 -- 172.21.15.70:0/1747654883 shutdown_connections 2024-02-10T07:39:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.281+0000 7fb350496700 1 -- 172.21.15.70:0/1747654883 >> 172.21.15.70:0/1747654883 conn(0x7fb348076e50 msgr2=0x7fb3480fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.281+0000 7fb350496700 1 -- 172.21.15.70:0/1747654883 shutdown_connections 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.281+0000 7fb350496700 1 -- 172.21.15.70:0/1747654883 wait complete. 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.281+0000 7fb350496700 1 Processor -- start 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.281+0000 7fb350496700 1 -- start start 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb350496700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb3481a6aa0 con 0x7fb348109860 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb350496700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb3481a7c80 con 0x7fb3481059e0 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb350496700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb3481a8e80 con 0x7fb34810d430 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb34da31700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb348109860 0x7fb3481a17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32968/0 (socket says 172.21.15.70:32968) 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb34ea33700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fb34810d430 0x7fb3481a5160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:43494/0 (socket says 172.21.15.70:43494) 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb34da31700 1 -- 172.21.15.70:0/3100381111 learned_addr learned my addr 172.21.15.70:0/3100381111 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808888031 0 0) 0x7fb3481a8e80 con 0x7fb34810d430 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3340035b0 con 0x7fb34810d430 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180931088 0 0) 0x7fb3481a6aa0 con 0x7fb348109860 2024-02-10T07:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3481a8e80 con 0x7fb348109860 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605515480 0 0) 0x7fb3481a7c80 con 0x7fb3481059e0 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3481a6aa0 con 0x7fb3481059e0 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1654854791 0 0) 0x7fb3340035b0 con 0x7fb34810d430 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3481a7c80 con 0x7fb34810d430 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb340004700 con 0x7fb34810d430 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1304850311 0 0) 0x7fb3481a8e80 con 0x7fb348109860 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3340035b0 con 0x7fb348109860 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3985701996 0 0) 0x7fb3481a6aa0 con 0x7fb3481059e0 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3481a8e80 con 0x7fb3481059e0 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb338001890 con 0x7fb348109860 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb344001830 con 0x7fb3481059e0 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2499698165 0 0) 0x7fb3481a7c80 con 0x7fb34810d430 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 >> v1:172.21.15.155:6789/0 conn(0x7fb3481059e0 legacy=0x7fb34810cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:06.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 >> v1:172.21.15.70:6789/0 conn(0x7fb348109860 legacy=0x7fb3481a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.282+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3481aa080 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.283+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3481a6c90 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.283+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3481a7110 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.283+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fb340001e70 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.283+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb3400050d0 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.284+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c005120 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.284+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fb340005230 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.285+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fb340088bd0 con 0x7fb34810d430 2024-02-10T07:39:06.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.289+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fb340056020 con 0x7fb34810d430 2024-02-10T07:39:06.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.492+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb32c0053d0 con 0x7fb34810d430 2024-02-10T07:39:06.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.492+0000 7fb33f7fe700 1 -- 172.21.15.70:0/3100381111 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (unknown 2131975755 0 616938113) 0x7fb340054390 con 0x7fb34810d430 2024-02-10T07:39:06.493 INFO:teuthology.orchestra.run.smithi070.stdout:167503724572 2024-02-10T07:39:06.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.494+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 >> v1:172.21.15.70:6800/4101821536 conn(0x7fb33406ea20 legacy=0x7fb334070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:06.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.494+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 >> v1:172.21.15.70:6790/0 conn(0x7fb34810d430 legacy=0x7fb3481a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:06.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.494+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 shutdown_connections 2024-02-10T07:39:06.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.494+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 >> 172.21.15.70:0/3100381111 conn(0x7fb348076e50 msgr2=0x7fb348106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:06.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.494+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 shutdown_connections 2024-02-10T07:39:06.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:06.494+0000 7fb350496700 1 -- 172.21.15.70:0/3100381111 wait complete. 2024-02-10T07:39:06.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:06 smithi070 ceph-mon[29548]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:06.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3100381111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T07:39:07.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574893 for osd.1 2024-02-10T07:39:07.209 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:07.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:06 smithi070 ceph-mon[39923]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:07.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3100381111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T07:39:07.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:06 smithi155 ceph-mon[32688]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:07.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3100381111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T07:39:07.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462930 got 201863462934 for osd.6 2024-02-10T07:39:07.316 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:07.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.716+0000 7fd71f532700 1 -- 172.21.15.70:0/4286295784 >> v1:172.21.15.70:6790/0 conn(0x7fd718114cd0 legacy=0x7fd7181170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.716+0000 7fd71f532700 1 -- 172.21.15.70:0/4286295784 shutdown_connections 2024-02-10T07:39:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.716+0000 7fd71f532700 1 -- 172.21.15.70:0/4286295784 >> 172.21.15.70:0/4286295784 conn(0x7fd7180fd600 msgr2=0x7fd7180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.716+0000 7fd71f532700 1 -- 172.21.15.70:0/4286295784 shutdown_connections 2024-02-10T07:39:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.716+0000 7fd71f532700 1 -- 172.21.15.70:0/4286295784 wait complete. 2024-02-10T07:39:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.716+0000 7fd71f532700 1 Processor -- start 2024-02-10T07:39:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd71f532700 1 -- start start 2024-02-10T07:39:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd71f532700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd7181acf70 con 0x7fd7180740c0 2024-02-10T07:39:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd71f532700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd7181ae150 con 0x7fd718074bb0 2024-02-10T07:39:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd71f532700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd7181af350 con 0x7fd718114cd0 2024-02-10T07:39:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd71d2ce700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd7180740c0 0x7fd7181143b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53072/0 (socket says 172.21.15.70:53072) 2024-02-10T07:39:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd71dacf700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fd718114cd0 0x7fd7181ab630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:47606/0 (socket says 172.21.15.70:47606) 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd71d2ce700 1 -- 172.21.15.70:0/3491199788 learned_addr learned my addr 172.21.15.70:0/3491199788 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547149427 0 0) 0x7fd7181acf70 con 0x7fd7180740c0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7040049e0 con 0x7fd7180740c0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762957984 0 0) 0x7fd7181af350 con 0x7fd718114cd0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7181acf70 con 0x7fd718114cd0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320945204 0 0) 0x7fd7181ae150 con 0x7fd718074bb0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7181af350 con 0x7fd718074bb0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 993314176 0 0) 0x7fd7040049e0 con 0x7fd7180740c0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7181ae150 con 0x7fd7180740c0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd714001930 con 0x7fd7180740c0 2024-02-10T07:39:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4214088936 0 0) 0x7fd7181acf70 con 0x7fd718114cd0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7040049e0 con 0x7fd718114cd0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd710004960 con 0x7fd718114cd0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.717+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1921311202 0 0) 0x7fd7181af350 con 0x7fd718074bb0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd7181acf70 con 0x7fd718074bb0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd7080018d0 con 0x7fd718074bb0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2494107195 0 0) 0x7fd7181ae150 con 0x7fd7180740c0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 >> v1:172.21.15.70:6790/0 conn(0x7fd718114cd0 legacy=0x7fd7181ab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 >> v1:172.21.15.155:6789/0 conn(0x7fd718074bb0 legacy=0x7fd7181a7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7181b0550 con 0x7fd7180740c0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7181af540 con 0x7fd7180740c0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7181af9c0 con 0x7fd7180740c0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fd714001cc0 con 0x7fd7180740c0 2024-02-10T07:39:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.718+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fd714005d30 con 0x7fd7180740c0 2024-02-10T07:39:07.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.719+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc005120 con 0x7fd7180740c0 2024-02-10T07:39:07.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.719+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fd714004070 con 0x7fd7180740c0 2024-02-10T07:39:07.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.720+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fd7140888b0 con 0x7fd7180740c0 2024-02-10T07:39:07.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.724+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fd714055c90 con 0x7fd7180740c0 2024-02-10T07:39:07.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.927+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd6fc0053d0 con 0x7fd7180740c0 2024-02-10T07:39:07.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.928+0000 7fd70e7fc700 1 -- 172.21.15.70:0/3491199788 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (unknown 383520633 0 1195175880) 0x7fd714054000 con 0x7fd7180740c0 2024-02-10T07:39:07.929 INFO:teuthology.orchestra.run.smithi070.stdout:111669149733 2024-02-10T07:39:07.930 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:39:07.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.931+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 >> v1:172.21.15.70:6800/4101821536 conn(0x7fd70406ea10 legacy=0x7fd704070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:07.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.931+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 >> v1:172.21.15.70:6789/0 conn(0x7fd7180740c0 legacy=0x7fd7181143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:07.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.931+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 shutdown_connections 2024-02-10T07:39:07.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.931+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 >> 172.21.15.70:0/3491199788 conn(0x7fd7180fd600 msgr2=0x7fd7180ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:07.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.931+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 shutdown_connections 2024-02-10T07:39:07.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:07.931+0000 7fd71f532700 1 -- 172.21.15.70:0/3491199788 wait complete. 2024-02-10T07:39:08.219 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:07 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:08.220 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3491199788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T07:39:08.220 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:07 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:08.220 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3491199788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T07:39:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:07 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3491199788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T07:39:09.186 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149733 for osd.3 2024-02-10T07:39:09.187 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:09.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:08 smithi070 ceph-mon[29548]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:09.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:08 smithi070 ceph-mon[39923]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:09.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:08 smithi155 ceph-mon[32688]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:09.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724567 got 167503724572 for osd.5 2024-02-10T07:39:09.628 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:09 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:10.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:09 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:10.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:09 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:10.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.566+0000 7fcf6a247700 1 -- 172.21.15.70:0/4237601925 >> v1:172.21.15.70:6789/0 conn(0x7fcf64109860 legacy=0x7fcf6410bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:10.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.566+0000 7fcf6a247700 1 -- 172.21.15.70:0/4237601925 shutdown_connections 2024-02-10T07:39:10.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.566+0000 7fcf6a247700 1 -- 172.21.15.70:0/4237601925 >> 172.21.15.70:0/4237601925 conn(0x7fcf640fc4c0 msgr2=0x7fcf640fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:10.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.566+0000 7fcf6a247700 1 -- 172.21.15.70:0/4237601925 shutdown_connections 2024-02-10T07:39:10.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.566+0000 7fcf6a247700 1 -- 172.21.15.70:0/4237601925 wait complete. 2024-02-10T07:39:10.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.566+0000 7fcf6a247700 1 Processor -- start 2024-02-10T07:39:10.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.566+0000 7fcf6a247700 1 -- start start 2024-02-10T07:39:10.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf6a247700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf641a6960 con 0x7fcf6410d430 2024-02-10T07:39:10.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf6a247700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf641a7b40 con 0x7fcf641069b0 2024-02-10T07:39:10.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf6a247700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf641a8d40 con 0x7fcf64109860 2024-02-10T07:39:10.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf62ffd700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fcf64109860 0x7fcf641a19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:47632/0 (socket says 172.21.15.70:47632) 2024-02-10T07:39:10.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf63fff700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcf6410d430 0x7fcf641a5110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53084/0 (socket says 172.21.15.70:53084) 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf62ffd700 1 -- 172.21.15.70:0/3658606396 learned_addr learned my addr 172.21.15.70:0/3658606396 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219932109 0 0) 0x7fcf641a6960 con 0x7fcf6410d430 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf4c0031f0 con 0x7fcf6410d430 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178015272 0 0) 0x7fcf641a8d40 con 0x7fcf64109860 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf641a6960 con 0x7fcf64109860 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958348332 0 0) 0x7fcf641a7b40 con 0x7fcf641069b0 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf641a8d40 con 0x7fcf641069b0 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3147155023 0 0) 0x7fcf4c0031f0 con 0x7fcf6410d430 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcf641a7b40 con 0x7fcf6410d430 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcf5c001bb0 con 0x7fcf6410d430 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2673071091 0 0) 0x7fcf641a6960 con 0x7fcf64109860 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.567+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcf4c0031f0 con 0x7fcf64109860 2024-02-10T07:39:10.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4074114392 0 0) 0x7fcf641a8d40 con 0x7fcf641069b0 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcf641a6960 con 0x7fcf641069b0 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcf580045b0 con 0x7fcf64109860 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcf54001850 con 0x7fcf641069b0 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1900962350 0 0) 0x7fcf641a7b40 con 0x7fcf6410d430 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 >> v1:172.21.15.70:6790/0 conn(0x7fcf64109860 legacy=0x7fcf641a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 >> v1:172.21.15.155:6789/0 conn(0x7fcf641069b0 legacy=0x7fcf6410ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf641a9f40 con 0x7fcf6410d430 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf641a8f30 con 0x7fcf6410d430 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.568+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf641a93b0 con 0x7fcf6410d430 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.569+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf641020f0 con 0x7fcf6410d430 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.569+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fcf5c003eb0 con 0x7fcf6410d430 2024-02-10T07:39:10.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.569+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcf5c005f40 con 0x7fcf6410d430 2024-02-10T07:39:10.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.569+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fcf5c01c690 con 0x7fcf6410d430 2024-02-10T07:39:10.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.570+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fcf5c089b60 con 0x7fcf6410d430 2024-02-10T07:39:10.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.574+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fcf5c056e00 con 0x7fcf6410d430 2024-02-10T07:39:10.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.773+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcf64109fd0 con 0x7fcf6410d430 2024-02-10T07:39:10.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.773+0000 7fcf60ff9700 1 -- 172.21.15.70:0/3658606396 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (unknown 1482059429 0 397825816) 0x7fcf5c055170 con 0x7fcf6410d430 2024-02-10T07:39:10.773 INFO:teuthology.orchestra.run.smithi070.stdout:236223201299 2024-02-10T07:39:10.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.775+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 >> v1:172.21.15.70:6800/4101821536 conn(0x7fcf4c06eb90 legacy=0x7fcf4c071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:10.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.775+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 >> v1:172.21.15.70:6789/0 conn(0x7fcf6410d430 legacy=0x7fcf641a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:10.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.775+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 shutdown_connections 2024-02-10T07:39:10.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.775+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 >> 172.21.15.70:0/3658606396 conn(0x7fcf640fc4c0 msgr2=0x7fcf641100a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:10.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.775+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 shutdown_connections 2024-02-10T07:39:10.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:10.775+0000 7fcf6a247700 1 -- 172.21.15.70:0/3658606396 wait complete. 2024-02-10T07:39:11.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201294 got 236223201299 for osd.7 2024-02-10T07:39:11.357 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:11.357 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-10T07:39:11.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T07:39:11.603 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:39:11.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[39923]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:11.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:11.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:11.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3658606396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T07:39:11.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[29548]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:11.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:11.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:11.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3658606396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T07:39:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:11 smithi155 ceph-mon[32688]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3658606396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T07:39:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:39:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:39:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.376+0000 7fb1620a2700 1 -- 172.21.15.70:0/2576716678 >> v1:172.21.15.70:6790/0 conn(0x7fb15c106ae0 legacy=0x7fb15c106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.376+0000 7fb1620a2700 1 -- 172.21.15.70:0/2576716678 shutdown_connections 2024-02-10T07:39:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.376+0000 7fb1620a2700 1 -- 172.21.15.70:0/2576716678 >> 172.21.15.70:0/2576716678 conn(0x7fb15c076e50 msgr2=0x7fb15c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.376+0000 7fb1620a2700 1 -- 172.21.15.70:0/2576716678 shutdown_connections 2024-02-10T07:39:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.376+0000 7fb1620a2700 1 -- 172.21.15.70:0/2576716678 wait complete. 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.376+0000 7fb1620a2700 1 Processor -- start 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb1620a2700 1 -- start start 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb1620a2700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb15c1a6900 con 0x7fb15c10e630 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb1620a2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb15c1a7ae0 con 0x7fb15c106ae0 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb15affd700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fb15c10a960 0x7fb15c1a19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:47650/0 (socket says 172.21.15.70:47650) 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb15affd700 1 -- 172.21.15.70:0/1145940833 learned_addr learned my addr 172.21.15.70:0/1145940833 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb15c1a8ce0 con 0x7fb15c10a960 2024-02-10T07:39:13.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95890603 0 0) 0x7fb15c1a7ae0 con 0x7fb15c106ae0 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1440035b0 con 0x7fb15c106ae0 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55986601 0 0) 0x7fb15c1a6900 con 0x7fb15c10e630 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.377+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb15c1a7ae0 con 0x7fb15c10e630 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780012358 0 0) 0x7fb15c1a8ce0 con 0x7fb15c10a960 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb15c1a6900 con 0x7fb15c10a960 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 193408844 0 0) 0x7fb1440035b0 con 0x7fb15c106ae0 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb15c1a8ce0 con 0x7fb15c106ae0 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb14c0015d0 con 0x7fb15c106ae0 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1969451409 0 0) 0x7fb15c1a7ae0 con 0x7fb15c10e630 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb1440035b0 con 0x7fb15c10e630 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb154001c30 con 0x7fb15c10e630 2024-02-10T07:39:13.380 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2840062084 0 0) 0x7fb15c1a6900 con 0x7fb15c10a960 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb15c1a7ae0 con 0x7fb15c10a960 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb1500017c0 con 0x7fb15c10a960 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4255421140 0 0) 0x7fb15c1a8ce0 con 0x7fb15c106ae0 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 >> v1:172.21.15.70:6790/0 conn(0x7fb15c10a960 legacy=0x7fb15c1a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 >> v1:172.21.15.70:6789/0 conn(0x7fb15c10e630 legacy=0x7fb15c1a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c1a7cb0 con 0x7fb15c106ae0 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c1a7f60 con 0x7fb15c106ae0 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c1a83e0 con 0x7fb15c106ae0 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fb14c001a90 con 0x7fb15c106ae0 2024-02-10T07:39:13.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.378+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fb14c004da0 con 0x7fb15c106ae0 2024-02-10T07:39:13.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.379+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb148005120 con 0x7fb15c106ae0 2024-02-10T07:39:13.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.379+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fb14c01b430 con 0x7fb15c106ae0 2024-02-10T07:39:13.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.380+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7fb14c087a10 con 0x7fb15c106ae0 2024-02-10T07:39:13.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.384+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fb14c054a20 con 0x7fb15c106ae0 2024-02-10T07:39:13.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.581+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 --> v1:172.21.15.70:6800/4101821536 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb148000b90 con 0x7fb144072f60 2024-02-10T07:39:13.582 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:13.582 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi070 to 3760M 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: Updating smithi070:/etc/ceph/ceph.conf 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: Updating smithi155:/etc/ceph/ceph.conf 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.583 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi070 to 3760M 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:39:13.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:39:13.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: Updating smithi070:/etc/ceph/ceph.conf 2024-02-10T07:39:13.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: Updating smithi155:/etc/ceph/ceph.conf 2024-02-10T07:39:13.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.585 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:13 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.604+0000 7fb158ff9700 1 -- 172.21.15.70:0/1145940833 <== mgr.34502 v1:172.21.15.70:6800/4101821536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384419 (unknown 2965378022 0 4237467849) 0x7fb148000b90 con 0x7fb144072f60 2024-02-10T07:39:13.610 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:39:13.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.608+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 >> v1:172.21.15.70:6800/4101821536 conn(0x7fb144072f60 legacy=0x7fb144075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:13.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.608+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 >> v1:172.21.15.155:6789/0 conn(0x7fb15c106ae0 legacy=0x7fb15c0feeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:13.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.608+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 shutdown_connections 2024-02-10T07:39:13.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.608+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 >> 172.21.15.70:0/1145940833 conn(0x7fb15c076e50 msgr2=0x7fb15c10d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:13.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.608+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 shutdown_connections 2024-02-10T07:39:13.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:13.608+0000 7fb1620a2700 1 -- 172.21.15.70:0/1145940833 wait complete. 2024-02-10T07:39:13.613 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-02-10T07:39:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi070 to 3760M 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: Updating smithi070:/etc/ceph/ceph.conf 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: Updating smithi155:/etc/ceph/ceph.conf 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:13 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:14.245 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-02-10T07:39:11.760448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":996,"num_read_kb":842,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":236080,"kb_used_data":7452,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725136,"statfs":{"total":767960285184,"available":767718539264,"internally_reserved":0,"allocated":7630848,"data_stored":4046444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002980"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747873+0000","last_change":"2024-02-10T07:38:04.186647+0000","last_active":"2024-02-10T07:38:37.747873+0000","last_peered":"2024-02-10T07:38:37.747873+0000","last_clean":"2024-02-10T07:38:37.747873+0000","last_became_active":"2024-02-10T07:38:04.186088+0000","last_became_peered":"2024-02-10T07:38:04.186088+0000","last_unstale":"2024-02-10T07:38:37.747873+0000","last_undegraded":"2024-02-10T07:38:37.747873+0000","last_fullsized":"2024-02-10T07:38:37.747873+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:21:31.816616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849094+0000","last_change":"2024-02-10T07:38:02.184576+0000","last_active":"2024-02-10T07:38:37.849094+0000","last_peered":"2024-02-10T07:38:37.849094+0000","last_clean":"2024-02-10T07:38:37.849094+0000","last_became_active":"2024-02-10T07:38:02.184185+0000","last_became_peered":"2024-02-10T07:38:02.184185+0000","last_unstale":"2024-02-10T07:38:37.849094+0000","last_undegraded":"2024-02-10T07:38:37.849094+0000","last_fullsized":"2024-02-10T07:38:37.849094+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:50:50.851110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849140+0000","last_change":"2024-02-10T07:37:58.158178+0000","last_active":"2024-02-10T07:38:37.849140+0000","last_peered":"2024-02-10T07:38:37.849140+0000","last_clean":"2024-02-10T07:38:37.849140+0000","last_became_active":"2024-02-10T07:37:58.157812+0000","last_became_peered":"2024-02-10T07:37:58.157812+0000","last_unstale":"2024-02-10T07:38:37.849140+0000","last_undegraded":"2024-02-10T07:38:37.849140+0000","last_fullsized":"2024-02-10T07:38:37.849140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:03:36.743454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747462+0000","last_change":"2024-02-10T07:38:00.162568+0000","last_active":"2024-02-10T07:38:37.747462+0000","last_peered":"2024-02-10T07:38:37.747462+0000","last_clean":"2024-02-10T07:38:37.747462+0000","last_became_active":"2024-02-10T07:38:00.162210+0000","last_became_peered":"2024-02-10T07:38:00.162210+0000","last_unstale":"2024-02-10T07:38:37.747462+0000","last_undegraded":"2024-02-10T07:38:37.747462+0000","last_fullsized":"2024-02-10T07:38:37.747462+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:03:39.071261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848975+0000","last_change":"2024-02-10T07:38:04.563853+0000","last_active":"2024-02-10T07:38:37.848975+0000","last_peered":"2024-02-10T07:38:37.848975+0000","last_clean":"2024-02-10T07:38:37.848975+0000","last_became_active":"2024-02-10T07:38:04.563512+0000","last_became_peered":"2024-02-10T07:38:04.563512+0000","last_unstale":"2024-02-10T07:38:37.848975+0000","last_undegraded":"2024-02-10T07:38:37.848975+0000","last_fullsized":"2024-02-10T07:38:37.848975+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:45.876177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747462+0000","last_change":"2024-02-10T07:38:02.185374+0000","last_active":"2024-02-10T07:38:37.747462+0000","last_peered":"2024-02-10T07:38:37.747462+0000","last_clean":"2024-02-10T07:38:37.747462+0000","last_became_active":"2024-02-10T07:38:02.184300+0000","last_became_peered":"2024-02-10T07:38:02.184300+0000","last_unstale":"2024-02-10T07:38:37.747462+0000","last_undegraded":"2024-02-10T07:38:37.747462+0000","last_fullsized":"2024-02-10T07:38:37.747462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:06:37.024983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851305+0000","last_change":"2024-02-10T07:37:58.154555+0000","last_active":"2024-02-10T07:38:37.851305+0000","last_peered":"2024-02-10T07:38:37.851305+0000","last_clean":"2024-02-10T07:38:37.851305+0000","last_became_active":"2024-02-10T07:37:58.154264+0000","last_became_peered":"2024-02-10T07:37:58.154264+0000","last_unstale":"2024-02-10T07:38:37.851305+0000","last_undegraded":"2024-02-10T07:38:37.851305+0000","last_fullsized":"2024-02-10T07:38:37.851305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:05:38.202189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849817+0000","last_change":"2024-02-10T07:38:00.168546+0000","last_active":"2024-02-10T07:38:37.849817+0000","last_peered":"2024-02-10T07:38:37.849817+0000","last_clean":"2024-02-10T07:38:37.849817+0000","last_became_active":"2024-02-10T07:38:00.168115+0000","last_became_peered":"2024-02-10T07:38:00.168115+0000","last_unstale":"2024-02-10T07:38:37.849817+0000","last_undegraded":"2024-02-10T07:38:37.849817+0000","last_fullsized":"2024-02-10T07:38:37.849817+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:37:54.192045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848145+0000","last_change":"2024-02-10T07:38:04.191353+0000","last_active":"2024-02-10T07:38:37.848145+0000","last_peered":"2024-02-10T07:38:37.848145+0000","last_clean":"2024-02-10T07:38:37.848145+0000","last_became_active":"2024-02-10T07:38:04.191014+0000","last_became_peered":"2024-02-10T07:38:04.191014+0000","last_unstale":"2024-02-10T07:38:37.848145+0000","last_undegraded":"2024-02-10T07:38:37.848145+0000","last_fullsized":"2024-02-10T07:38:37.848145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:58:43.576706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834650+0000","last_change":"2024-02-10T07:38:02.185353+0000","last_active":"2024-02-10T07:38:37.834650+0000","last_peered":"2024-02-10T07:38:37.834650+0000","last_clean":"2024-02-10T07:38:37.834650+0000","last_became_active":"2024-02-10T07:38:02.184968+0000","last_became_peered":"2024-02-10T07:38:02.184968+0000","last_unstale":"2024-02-10T07:38:37.834650+0000","last_undegraded":"2024-02-10T07:38:37.834650+0000","last_fullsized":"2024-02-10T07:38:37.834650+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:04:41.595322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834740+0000","last_change":"2024-02-10T07:37:58.153195+0000","last_active":"2024-02-10T07:38:37.834740+0000","last_peered":"2024-02-10T07:38:37.834740+0000","last_clean":"2024-02-10T07:38:37.834740+0000","last_became_active":"2024-02-10T07:37:58.152916+0000","last_became_peered":"2024-02-10T07:37:58.152916+0000","last_unstale":"2024-02-10T07:38:37.834740+0000","last_undegraded":"2024-02-10T07:38:37.834740+0000","last_fullsized":"2024-02-10T07:38:37.834740+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:20:15.789480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748113+0000","last_change":"2024-02-10T07:38:00.164083+0000","last_active":"2024-02-10T07:38:37.748113+0000","last_peered":"2024-02-10T07:38:37.748113+0000","last_clean":"2024-02-10T07:38:37.748113+0000","last_became_active":"2024-02-10T07:38:00.163762+0000","last_became_peered":"2024-02-10T07:38:00.163762+0000","last_unstale":"2024-02-10T07:38:37.748113+0000","last_undegraded":"2024-02-10T07:38:37.748113+0000","last_fullsized":"2024-02-10T07:38:37.748113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:33:52.219839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834294+0000","last_change":"2024-02-10T07:38:04.193340+0000","last_active":"2024-02-10T07:38:37.834294+0000","last_peered":"2024-02-10T07:38:37.834294+0000","last_clean":"2024-02-10T07:38:37.834294+0000","last_became_active":"2024-02-10T07:38:04.193003+0000","last_became_peered":"2024-02-10T07:38:04.193003+0000","last_unstale":"2024-02-10T07:38:37.834294+0000","last_undegraded":"2024-02-10T07:38:37.834294+0000","last_fullsized":"2024-02-10T07:38:37.834294+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:27:01.077971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844052+0000","last_change":"2024-02-10T07:38:02.186017+0000","last_active":"2024-02-10T07:38:37.844052+0000","last_peered":"2024-02-10T07:38:37.844052+0000","last_clean":"2024-02-10T07:38:37.844052+0000","last_became_active":"2024-02-10T07:38:02.185713+0000","last_became_peered":"2024-02-10T07:38:02.185713+0000","last_unstale":"2024-02-10T07:38:37.844052+0000","last_undegraded":"2024-02-10T07:38:37.844052+0000","last_fullsized":"2024-02-10T07:38:37.844052+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:19:31.424409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834664+0000","last_change":"2024-02-10T07:37:58.150762+0000","last_active":"2024-02-10T07:38:37.834664+0000","last_peered":"2024-02-10T07:38:37.834664+0000","last_clean":"2024-02-10T07:38:37.834664+0000","last_became_active":"2024-02-10T07:37:58.150477+0000","last_became_peered":"2024-02-10T07:37:58.150477+0000","last_unstale":"2024-02-10T07:38:37.834664+0000","last_undegraded":"2024-02-10T07:38:37.834664+0000","last_fullsized":"2024-02-10T07:38:37.834664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:44:47.243565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"69'5","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748915+0000","last_change":"2024-02-10T07:38:00.169434+0000","last_active":"2024-02-10T07:38:37.748915+0000","last_peered":"2024-02-10T07:38:37.748915+0000","last_clean":"2024-02-10T07:38:37.748915+0000","last_became_active":"2024-02-10T07:38:00.169148+0000","last_became_peered":"2024-02-10T07:38:00.169148+0000","last_unstale":"2024-02-10T07:38:37.748915+0000","last_undegraded":"2024-02-10T07:38:37.748915+0000","last_fullsized":"2024-02-10T07:38:37.748915+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:18.133325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.750255+0000","last_change":"2024-02-10T07:38:04.561736+0000","last_active":"2024-02-10T07:38:37.750255+0000","last_peered":"2024-02-10T07:38:37.750255+0000","last_clean":"2024-02-10T07:38:37.750255+0000","last_became_active":"2024-02-10T07:38:04.561400+0000","last_became_peered":"2024-02-10T07:38:04.561400+0000","last_unstale":"2024-02-10T07:38:37.750255+0000","last_undegraded":"2024-02-10T07:38:37.750255+0000","last_fullsized":"2024-02-10T07:38:37.750255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:32:05.994039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848395+0000","last_change":"2024-02-10T07:38:02.183003+0000","last_active":"2024-02-10T07:38:37.848395+0000","last_peered":"2024-02-10T07:38:37.848395+0000","last_clean":"2024-02-10T07:38:37.848395+0000","last_became_active":"2024-02-10T07:38:02.182700+0000","last_became_peered":"2024-02-10T07:38:02.182700+0000","last_unstale":"2024-02-10T07:38:37.848395+0000","last_undegraded":"2024-02-10T07:38:37.848395+0000","last_fullsized":"2024-02-10T07:38:37.848395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:23:19.764457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850257+0000","last_change":"2024-02-10T07:37:58.158806+0000","last_active":"2024-02-10T07:38:37.850257+0000","last_peered":"2024-02-10T07:38:37.850257+0000","last_clean":"2024-02-10T07:38:37.850257+0000","last_became_active":"2024-02-10T07:37:58.158534+0000","last_became_peered":"2024-02-10T07:37:58.158534+0000","last_unstale":"2024-02-10T07:38:37.850257+0000","last_undegraded":"2024-02-10T07:38:37.850257+0000","last_fullsized":"2024-02-10T07:38:37.850257+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:31:26.321110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745044+0000","last_change":"2024-02-10T07:38:00.170213+0000","last_active":"2024-02-10T07:38:37.745044+0000","last_peered":"2024-02-10T07:38:37.745044+0000","last_clean":"2024-02-10T07:38:37.745044+0000","last_became_active":"2024-02-10T07:38:00.169807+0000","last_became_peered":"2024-02-10T07:38:00.169807+0000","last_unstale":"2024-02-10T07:38:37.745044+0000","last_undegraded":"2024-02-10T07:38:37.745044+0000","last_fullsized":"2024-02-10T07:38:37.745044+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:15:47.790155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851845+0000","last_change":"2024-02-10T07:38:04.563035+0000","last_active":"2024-02-10T07:38:37.851845+0000","last_peered":"2024-02-10T07:38:37.851845+0000","last_clean":"2024-02-10T07:38:37.851845+0000","last_became_active":"2024-02-10T07:38:04.562573+0000","last_became_peered":"2024-02-10T07:38:04.562573+0000","last_unstale":"2024-02-10T07:38:37.851845+0000","last_undegraded":"2024-02-10T07:38:37.851845+0000","last_fullsized":"2024-02-10T07:38:37.851845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:13:10.575665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748177+0000","last_change":"2024-02-10T07:38:02.186897+0000","last_active":"2024-02-10T07:38:37.748177+0000","last_peered":"2024-02-10T07:38:37.748177+0000","last_clean":"2024-02-10T07:38:37.748177+0000","last_became_active":"2024-02-10T07:38:02.186606+0000","last_became_peered":"2024-02-10T07:38:02.186606+0000","last_unstale":"2024-02-10T07:38:37.748177+0000","last_undegraded":"2024-02-10T07:38:37.748177+0000","last_fullsized":"2024-02-10T07:38:37.748177+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:30:58.044937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748225+0000","last_change":"2024-02-10T07:37:58.157435+0000","last_active":"2024-02-10T07:38:37.748225+0000","last_peered":"2024-02-10T07:38:37.748225+0000","last_clean":"2024-02-10T07:38:37.748225+0000","last_became_active":"2024-02-10T07:37:58.157142+0000","last_became_peered":"2024-02-10T07:37:58.157142+0000","last_unstale":"2024-02-10T07:38:37.748225+0000","last_undegraded":"2024-02-10T07:38:37.748225+0000","last_fullsized":"2024-02-10T07:38:37.748225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:12.624671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"69'12","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850472+0000","last_change":"2024-02-10T07:38:00.169085+0000","last_active":"2024-02-10T07:38:37.850472+0000","last_peered":"2024-02-10T07:38:37.850472+0000","last_clean":"2024-02-10T07:38:37.850472+0000","last_became_active":"2024-02-10T07:38:00.168028+0000","last_became_peered":"2024-02-10T07:38:00.168028+0000","last_unstale":"2024-02-10T07:38:37.850472+0000","last_undegraded":"2024-02-10T07:38:37.850472+0000","last_fullsized":"2024-02-10T07:38:37.850472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:18:00.568320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844353+0000","last_change":"2024-02-10T07:38:04.189744+0000","last_active":"2024-02-10T07:38:37.844353+0000","last_peered":"2024-02-10T07:38:37.844353+0000","last_clean":"2024-02-10T07:38:37.844353+0000","last_became_active":"2024-02-10T07:38:04.189496+0000","last_became_peered":"2024-02-10T07:38:04.189496+0000","last_unstale":"2024-02-10T07:38:37.844353+0000","last_undegraded":"2024-02-10T07:38:37.844353+0000","last_fullsized":"2024-02-10T07:38:37.844353+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:12.837478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":51,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210809+0000","last_change":"2024-02-10T07:38:02.187005+0000","last_active":"2024-02-10T07:39:03.210809+0000","last_peered":"2024-02-10T07:39:03.210809+0000","last_clean":"2024-02-10T07:39:03.210809+0000","last_became_active":"2024-02-10T07:38:02.186671+0000","last_became_peered":"2024-02-10T07:38:02.186671+0000","last_unstale":"2024-02-10T07:39:03.210809+0000","last_undegraded":"2024-02-10T07:39:03.210809+0000","last_fullsized":"2024-02-10T07:39:03.210809+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:21.033127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849175+0000","last_change":"2024-02-10T07:37:58.155111+0000","last_active":"2024-02-10T07:38:37.849175+0000","last_peered":"2024-02-10T07:38:37.849175+0000","last_clean":"2024-02-10T07:38:37.849175+0000","last_became_active":"2024-02-10T07:37:58.154702+0000","last_became_peered":"2024-02-10T07:37:58.154702+0000","last_unstale":"2024-02-10T07:38:37.849175+0000","last_undegraded":"2024-02-10T07:38:37.849175+0000","last_fullsized":"2024-02-10T07:38:37.849175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:50:38.943235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849587+0000","last_change":"2024-02-10T07:38:00.767811+0000","last_active":"2024-02-10T07:38:37.849587+0000","last_peered":"2024-02-10T07:38:37.849587+0000","last_clean":"2024-02-10T07:38:37.849587+0000","last_became_active":"2024-02-10T07:38:00.767522+0000","last_became_peered":"2024-02-10T07:38:00.767522+0000","last_unstale":"2024-02-10T07:38:37.849587+0000","last_undegraded":"2024-02-10T07:38:37.849587+0000","last_fullsized":"2024-02-10T07:38:37.849587+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:31:21.312717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848518+0000","last_change":"2024-02-10T07:38:04.187165+0000","last_active":"2024-02-10T07:38:37.848518+0000","last_peered":"2024-02-10T07:38:37.848518+0000","last_clean":"2024-02-10T07:38:37.848518+0000","last_became_active":"2024-02-10T07:38:04.186402+0000","last_became_peered":"2024-02-10T07:38:04.186402+0000","last_unstale":"2024-02-10T07:38:37.848518+0000","last_undegraded":"2024-02-10T07:38:37.848518+0000","last_fullsized":"2024-02-10T07:38:37.848518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:07:33.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849905+0000","last_change":"2024-02-10T07:38:02.188677+0000","last_active":"2024-02-10T07:38:37.849905+0000","last_peered":"2024-02-10T07:38:37.849905+0000","last_clean":"2024-02-10T07:38:37.849905+0000","last_became_active":"2024-02-10T07:38:02.188391+0000","last_became_peered":"2024-02-10T07:38:02.188391+0000","last_unstale":"2024-02-10T07:38:37.849905+0000","last_undegraded":"2024-02-10T07:38:37.849905+0000","last_fullsized":"2024-02-10T07:38:37.849905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:49:27.901890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847848+0000","last_change":"2024-02-10T07:37:58.151815+0000","last_active":"2024-02-10T07:38:37.847848+0000","last_peered":"2024-02-10T07:38:37.847848+0000","last_clean":"2024-02-10T07:38:37.847848+0000","last_became_active":"2024-02-10T07:37:58.151572+0000","last_became_peered":"2024-02-10T07:37:58.151572+0000","last_unstale":"2024-02-10T07:38:37.847848+0000","last_undegraded":"2024-02-10T07:38:37.847848+0000","last_fullsized":"2024-02-10T07:38:37.847848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:57:09.209275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"69'11","reported_seq":66,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847970+0000","last_change":"2024-02-10T07:38:28.787238+0000","last_active":"2024-02-10T07:38:37.847970+0000","last_peered":"2024-02-10T07:38:37.847970+0000","last_clean":"2024-02-10T07:38:37.847970+0000","last_became_active":"2024-02-10T07:38:28.776345+0000","last_became_peered":"2024-02-10T07:38:28.776345+0000","last_unstale":"2024-02-10T07:38:37.847970+0000","last_undegraded":"2024-02-10T07:38:37.847970+0000","last_fullsized":"2024-02-10T07:38:37.847970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:55:48.034095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.843999+0000","last_change":"2024-02-10T07:38:04.189394+0000","last_active":"2024-02-10T07:38:37.843999+0000","last_peered":"2024-02-10T07:38:37.843999+0000","last_clean":"2024-02-10T07:38:37.843999+0000","last_became_active":"2024-02-10T07:38:04.189126+0000","last_became_peered":"2024-02-10T07:38:04.189126+0000","last_unstale":"2024-02-10T07:38:37.843999+0000","last_undegraded":"2024-02-10T07:38:37.843999+0000","last_fullsized":"2024-02-10T07:38:37.843999+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:06:49.453471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847377+0000","last_change":"2024-02-10T07:38:02.183392+0000","last_active":"2024-02-10T07:38:37.847377+0000","last_peered":"2024-02-10T07:38:37.847377+0000","last_clean":"2024-02-10T07:38:37.847377+0000","last_became_active":"2024-02-10T07:38:02.183011+0000","last_became_peered":"2024-02-10T07:38:02.183011+0000","last_unstale":"2024-02-10T07:38:37.847377+0000","last_undegraded":"2024-02-10T07:38:37.847377+0000","last_fullsized":"2024-02-10T07:38:37.847377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:37:04.557621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849039+0000","last_change":"2024-02-10T07:37:58.153169+0000","last_active":"2024-02-10T07:38:37.849039+0000","last_peered":"2024-02-10T07:38:37.849039+0000","last_clean":"2024-02-10T07:38:37.849039+0000","last_became_active":"2024-02-10T07:37:58.152890+0000","last_became_peered":"2024-02-10T07:37:58.152890+0000","last_unstale":"2024-02-10T07:38:37.849039+0000","last_undegraded":"2024-02-10T07:38:37.849039+0000","last_fullsized":"2024-02-10T07:38:37.849039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:50:21.140869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847953+0000","last_change":"2024-02-10T07:38:00.172054+0000","last_active":"2024-02-10T07:38:37.847953+0000","last_peered":"2024-02-10T07:38:37.847953+0000","last_clean":"2024-02-10T07:38:37.847953+0000","last_became_active":"2024-02-10T07:38:00.171747+0000","last_became_peered":"2024-02-10T07:38:00.171747+0000","last_unstale":"2024-02-10T07:38:37.847953+0000","last_undegraded":"2024-02-10T07:38:37.847953+0000","last_fullsized":"2024-02-10T07:38:37.847953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:29:06.097734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848792+0000","last_change":"2024-02-10T07:38:04.562960+0000","last_active":"2024-02-10T07:38:37.848792+0000","last_peered":"2024-02-10T07:38:37.848792+0000","last_clean":"2024-02-10T07:38:37.848792+0000","last_became_active":"2024-02-10T07:38:04.562557+0000","last_became_peered":"2024-02-10T07:38:04.562557+0000","last_unstale":"2024-02-10T07:38:37.848792+0000","last_undegraded":"2024-02-10T07:38:37.848792+0000","last_fullsized":"2024-02-10T07:38:37.848792+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:46:44.975257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844207+0000","last_change":"2024-02-10T07:38:02.190989+0000","last_active":"2024-02-10T07:38:37.844207+0000","last_peered":"2024-02-10T07:38:37.844207+0000","last_clean":"2024-02-10T07:38:37.844207+0000","last_became_active":"2024-02-10T07:38:02.190505+0000","last_became_peered":"2024-02-10T07:38:02.190505+0000","last_unstale":"2024-02-10T07:38:37.844207+0000","last_undegraded":"2024-02-10T07:38:37.844207+0000","last_fullsized":"2024-02-10T07:38:37.844207+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:09:43.496965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834954+0000","last_change":"2024-02-10T07:37:58.150118+0000","last_active":"2024-02-10T07:38:37.834954+0000","last_peered":"2024-02-10T07:38:37.834954+0000","last_clean":"2024-02-10T07:38:37.834954+0000","last_became_active":"2024-02-10T07:37:58.149812+0000","last_became_peered":"2024-02-10T07:37:58.149812+0000","last_unstale":"2024-02-10T07:38:37.834954+0000","last_undegraded":"2024-02-10T07:38:37.834954+0000","last_fullsized":"2024-02-10T07:38:37.834954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:24:20.526839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849672+0000","last_change":"2024-02-10T07:38:00.765844+0000","last_active":"2024-02-10T07:38:37.849672+0000","last_peered":"2024-02-10T07:38:37.849672+0000","last_clean":"2024-02-10T07:38:37.849672+0000","last_became_active":"2024-02-10T07:38:00.765467+0000","last_became_peered":"2024-02-10T07:38:00.765467+0000","last_unstale":"2024-02-10T07:38:37.849672+0000","last_undegraded":"2024-02-10T07:38:37.849672+0000","last_fullsized":"2024-02-10T07:38:37.849672+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:14:19.211109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847715+0000","last_change":"2024-02-10T07:38:04.190076+0000","last_active":"2024-02-10T07:38:37.847715+0000","last_peered":"2024-02-10T07:38:37.847715+0000","last_clean":"2024-02-10T07:38:37.847715+0000","last_became_active":"2024-02-10T07:38:04.189670+0000","last_became_peered":"2024-02-10T07:38:04.189670+0000","last_unstale":"2024-02-10T07:38:37.847715+0000","last_undegraded":"2024-02-10T07:38:37.847715+0000","last_fullsized":"2024-02-10T07:38:37.847715+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:42:20.254668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849059+0000","last_change":"2024-02-10T07:38:02.188571+0000","last_active":"2024-02-10T07:38:37.849059+0000","last_peered":"2024-02-10T07:38:37.849059+0000","last_clean":"2024-02-10T07:38:37.849059+0000","last_became_active":"2024-02-10T07:38:02.188246+0000","last_became_peered":"2024-02-10T07:38:02.188246+0000","last_unstale":"2024-02-10T07:38:37.849059+0000","last_undegraded":"2024-02-10T07:38:37.849059+0000","last_fullsized":"2024-02-10T07:38:37.849059+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:32:33.117903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844509+0000","last_change":"2024-02-10T07:37:58.153978+0000","last_active":"2024-02-10T07:38:37.844509+0000","last_peered":"2024-02-10T07:38:37.844509+0000","last_clean":"2024-02-10T07:38:37.844509+0000","last_became_active":"2024-02-10T07:37:58.153407+0000","last_became_peered":"2024-02-10T07:37:58.153407+0000","last_unstale":"2024-02-10T07:38:37.844509+0000","last_undegraded":"2024-02-10T07:38:37.844509+0000","last_fullsized":"2024-02-10T07:38:37.844509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:40:28.719385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848397+0000","last_change":"2024-02-10T07:38:00.164466+0000","last_active":"2024-02-10T07:38:37.848397+0000","last_peered":"2024-02-10T07:38:37.848397+0000","last_clean":"2024-02-10T07:38:37.848397+0000","last_became_active":"2024-02-10T07:38:00.162718+0000","last_became_peered":"2024-02-10T07:38:00.162718+0000","last_unstale":"2024-02-10T07:38:37.848397+0000","last_undegraded":"2024-02-10T07:38:37.848397+0000","last_fullsized":"2024-02-10T07:38:37.848397+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:15:16.350207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849363+0000","last_change":"2024-02-10T07:38:04.197091+0000","last_active":"2024-02-10T07:38:37.849363+0000","last_peered":"2024-02-10T07:38:37.849363+0000","last_clean":"2024-02-10T07:38:37.849363+0000","last_became_active":"2024-02-10T07:38:04.196773+0000","last_became_peered":"2024-02-10T07:38:04.196773+0000","last_unstale":"2024-02-10T07:38:37.849363+0000","last_undegraded":"2024-02-10T07:38:37.849363+0000","last_fullsized":"2024-02-10T07:38:37.849363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:46:54.401235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848312+0000","last_change":"2024-02-10T07:38:02.185433+0000","last_active":"2024-02-10T07:38:37.848312+0000","last_peered":"2024-02-10T07:38:37.848312+0000","last_clean":"2024-02-10T07:38:37.848312+0000","last_became_active":"2024-02-10T07:38:02.185111+0000","last_became_peered":"2024-02-10T07:38:02.185111+0000","last_unstale":"2024-02-10T07:38:37.848312+0000","last_undegraded":"2024-02-10T07:38:37.848312+0000","last_fullsized":"2024-02-10T07:38:37.848312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:09.343575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748416+0000","last_change":"2024-02-10T07:37:58.152079+0000","last_active":"2024-02-10T07:38:37.748416+0000","last_peered":"2024-02-10T07:38:37.748416+0000","last_clean":"2024-02-10T07:38:37.748416+0000","last_became_active":"2024-02-10T07:37:58.151811+0000","last_became_peered":"2024-02-10T07:37:58.151811+0000","last_unstale":"2024-02-10T07:38:37.748416+0000","last_undegraded":"2024-02-10T07:38:37.748416+0000","last_fullsized":"2024-02-10T07:38:37.748416+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:48:42.546171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748745+0000","last_change":"2024-02-10T07:38:00.168712+0000","last_active":"2024-02-10T07:38:37.748745+0000","last_peered":"2024-02-10T07:38:37.748745+0000","last_clean":"2024-02-10T07:38:37.748745+0000","last_became_active":"2024-02-10T07:38:00.167745+0000","last_became_peered":"2024-02-10T07:38:00.167745+0000","last_unstale":"2024-02-10T07:38:37.748745+0000","last_undegraded":"2024-02-10T07:38:37.748745+0000","last_fullsized":"2024-02-10T07:38:37.748745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:47:11.415857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849273+0000","last_change":"2024-02-10T07:38:04.197103+0000","last_active":"2024-02-10T07:38:37.849273+0000","last_peered":"2024-02-10T07:38:37.849273+0000","last_clean":"2024-02-10T07:38:37.849273+0000","last_became_active":"2024-02-10T07:38:04.196769+0000","last_became_peered":"2024-02-10T07:38:04.196769+0000","last_unstale":"2024-02-10T07:38:37.849273+0000","last_undegraded":"2024-02-10T07:38:37.849273+0000","last_fullsized":"2024-02-10T07:38:37.849273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:54:44.258613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210815+0000","last_change":"2024-02-10T07:38:02.185745+0000","last_active":"2024-02-10T07:39:03.210815+0000","last_peered":"2024-02-10T07:39:03.210815+0000","last_clean":"2024-02-10T07:39:03.210815+0000","last_became_active":"2024-02-10T07:38:02.185473+0000","last_became_peered":"2024-02-10T07:38:02.185473+0000","last_unstale":"2024-02-10T07:39:03.210815+0000","last_undegraded":"2024-02-10T07:39:03.210815+0000","last_fullsized":"2024-02-10T07:39:03.210815+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:36:59.933516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844133+0000","last_change":"2024-02-10T07:37:58.155405+0000","last_active":"2024-02-10T07:38:37.844133+0000","last_peered":"2024-02-10T07:38:37.844133+0000","last_clean":"2024-02-10T07:38:37.844133+0000","last_became_active":"2024-02-10T07:37:58.155112+0000","last_became_peered":"2024-02-10T07:37:58.155112+0000","last_unstale":"2024-02-10T07:38:37.844133+0000","last_undegraded":"2024-02-10T07:38:37.844133+0000","last_fullsized":"2024-02-10T07:38:37.844133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:34:44.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850742+0000","last_change":"2024-02-10T07:38:00.765860+0000","last_active":"2024-02-10T07:38:37.850742+0000","last_peered":"2024-02-10T07:38:37.850742+0000","last_clean":"2024-02-10T07:38:37.850742+0000","last_became_active":"2024-02-10T07:38:00.765550+0000","last_became_peered":"2024-02-10T07:38:00.765550+0000","last_unstale":"2024-02-10T07:38:37.850742+0000","last_undegraded":"2024-02-10T07:38:37.850742+0000","last_fullsized":"2024-02-10T07:38:37.850742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:18:57.658269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.749941+0000","last_change":"2024-02-10T07:38:04.189975+0000","last_active":"2024-02-10T07:38:37.749941+0000","last_peered":"2024-02-10T07:38:37.749941+0000","last_clean":"2024-02-10T07:38:37.749941+0000","last_became_active":"2024-02-10T07:38:04.189674+0000","last_became_peered":"2024-02-10T07:38:04.189674+0000","last_unstale":"2024-02-10T07:38:37.749941+0000","last_undegraded":"2024-02-10T07:38:37.749941+0000","last_fullsized":"2024-02-10T07:38:37.749941+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:59:42.747712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210606+0000","last_change":"2024-02-10T07:38:02.187629+0000","last_active":"2024-02-10T07:39:03.210606+0000","last_peered":"2024-02-10T07:39:03.210606+0000","last_clean":"2024-02-10T07:39:03.210606+0000","last_became_active":"2024-02-10T07:38:02.187277+0000","last_became_peered":"2024-02-10T07:38:02.187277+0000","last_unstale":"2024-02-10T07:39:03.210606+0000","last_undegraded":"2024-02-10T07:39:03.210606+0000","last_fullsized":"2024-02-10T07:39:03.210606+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:54:07.636346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848402+0000","last_change":"2024-02-10T07:37:58.159222+0000","last_active":"2024-02-10T07:38:37.848402+0000","last_peered":"2024-02-10T07:38:37.848402+0000","last_clean":"2024-02-10T07:38:37.848402+0000","last_became_active":"2024-02-10T07:37:58.158523+0000","last_became_peered":"2024-02-10T07:37:58.158523+0000","last_unstale":"2024-02-10T07:38:37.848402+0000","last_undegraded":"2024-02-10T07:38:37.848402+0000","last_fullsized":"2024-02-10T07:38:37.848402+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:08:28.933514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.835050+0000","last_change":"2024-02-10T07:38:00.768575+0000","last_active":"2024-02-10T07:38:37.835050+0000","last_peered":"2024-02-10T07:38:37.835050+0000","last_clean":"2024-02-10T07:38:37.835050+0000","last_became_active":"2024-02-10T07:38:00.768184+0000","last_became_peered":"2024-02-10T07:38:00.768184+0000","last_unstale":"2024-02-10T07:38:37.835050+0000","last_undegraded":"2024-02-10T07:38:37.835050+0000","last_fullsized":"2024-02-10T07:38:37.835050+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:18:34.337454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.746049+0000","last_change":"2024-02-10T07:38:04.192255+0000","last_active":"2024-02-10T07:38:37.746049+0000","last_peered":"2024-02-10T07:38:37.746049+0000","last_clean":"2024-02-10T07:38:37.746049+0000","last_became_active":"2024-02-10T07:38:04.191959+0000","last_became_peered":"2024-02-10T07:38:04.191959+0000","last_unstale":"2024-02-10T07:38:37.746049+0000","last_undegraded":"2024-02-10T07:38:37.746049+0000","last_fullsized":"2024-02-10T07:38:37.746049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:47:40.619894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851740+0000","last_change":"2024-02-10T07:38:02.188897+0000","last_active":"2024-02-10T07:38:37.851740+0000","last_peered":"2024-02-10T07:38:37.851740+0000","last_clean":"2024-02-10T07:38:37.851740+0000","last_became_active":"2024-02-10T07:38:02.188576+0000","last_became_peered":"2024-02-10T07:38:02.188576+0000","last_unstale":"2024-02-10T07:38:37.851740+0000","last_undegraded":"2024-02-10T07:38:37.851740+0000","last_fullsized":"2024-02-10T07:38:37.851740+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:11:27.500027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844069+0000","last_change":"2024-02-10T07:37:58.155052+0000","last_active":"2024-02-10T07:38:37.844069+0000","last_peered":"2024-02-10T07:38:37.844069+0000","last_clean":"2024-02-10T07:38:37.844069+0000","last_became_active":"2024-02-10T07:37:58.154767+0000","last_became_peered":"2024-02-10T07:37:58.154767+0000","last_unstale":"2024-02-10T07:38:37.844069+0000","last_undegraded":"2024-02-10T07:38:37.844069+0000","last_fullsized":"2024-02-10T07:38:37.844069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:32:26.672068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849712+0000","last_change":"2024-02-10T07:38:00.767800+0000","last_active":"2024-02-10T07:38:37.849712+0000","last_peered":"2024-02-10T07:38:37.849712+0000","last_clean":"2024-02-10T07:38:37.849712+0000","last_became_active":"2024-02-10T07:38:00.767513+0000","last_became_peered":"2024-02-10T07:38:00.767513+0000","last_unstale":"2024-02-10T07:38:37.849712+0000","last_undegraded":"2024-02-10T07:38:37.849712+0000","last_fullsized":"2024-02-10T07:38:37.849712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:13:09.563969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844455+0000","last_change":"2024-02-10T07:38:04.564998+0000","last_active":"2024-02-10T07:38:37.844455+0000","last_peered":"2024-02-10T07:38:37.844455+0000","last_clean":"2024-02-10T07:38:37.844455+0000","last_became_active":"2024-02-10T07:38:04.564088+0000","last_became_peered":"2024-02-10T07:38:04.564088+0000","last_unstale":"2024-02-10T07:38:37.844455+0000","last_undegraded":"2024-02-10T07:38:37.844455+0000","last_fullsized":"2024-02-10T07:38:37.844455+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:52:22.234034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834953+0000","last_change":"2024-02-10T07:38:02.184168+0000","last_active":"2024-02-10T07:38:37.834953+0000","last_peered":"2024-02-10T07:38:37.834953+0000","last_clean":"2024-02-10T07:38:37.834953+0000","last_became_active":"2024-02-10T07:38:02.182892+0000","last_became_peered":"2024-02-10T07:38:02.182892+0000","last_unstale":"2024-02-10T07:38:37.834953+0000","last_undegraded":"2024-02-10T07:38:37.834953+0000","last_fullsized":"2024-02-10T07:38:37.834953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:06:44.551793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847320+0000","last_change":"2024-02-10T07:37:58.150290+0000","last_active":"2024-02-10T07:38:37.847320+0000","last_peered":"2024-02-10T07:38:37.847320+0000","last_clean":"2024-02-10T07:38:37.847320+0000","last_became_active":"2024-02-10T07:37:58.149988+0000","last_became_peered":"2024-02-10T07:37:58.149988+0000","last_unstale":"2024-02-10T07:38:37.847320+0000","last_undegraded":"2024-02-10T07:38:37.847320+0000","last_fullsized":"2024-02-10T07:38:37.847320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:08:59.591165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"69'6","reported_seq":38,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851851+0000","last_change":"2024-02-10T07:38:00.765838+0000","last_active":"2024-02-10T07:38:37.851851+0000","last_peered":"2024-02-10T07:38:37.851851+0000","last_clean":"2024-02-10T07:38:37.851851+0000","last_became_active":"2024-02-10T07:38:00.765455+0000","last_became_peered":"2024-02-10T07:38:00.765455+0000","last_unstale":"2024-02-10T07:38:37.851851+0000","last_undegraded":"2024-02-10T07:38:37.851851+0000","last_fullsized":"2024-02-10T07:38:37.851851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:50.099879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834686+0000","last_change":"2024-02-10T07:38:04.564345+0000","last_active":"2024-02-10T07:38:37.834686+0000","last_peered":"2024-02-10T07:38:37.834686+0000","last_clean":"2024-02-10T07:38:37.834686+0000","last_became_active":"2024-02-10T07:38:04.563522+0000","last_became_peered":"2024-02-10T07:38:04.563522+0000","last_unstale":"2024-02-10T07:38:37.834686+0000","last_undegraded":"2024-02-10T07:38:37.834686+0000","last_fullsized":"2024-02-10T07:38:37.834686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:41:36.968641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"62'2","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844658+0000","last_change":"2024-02-10T07:37:58.154342+0000","last_active":"2024-02-10T07:38:37.844658+0000","last_peered":"2024-02-10T07:38:37.844658+0000","last_clean":"2024-02-10T07:38:37.844658+0000","last_became_active":"2024-02-10T07:37:58.154083+0000","last_became_peered":"2024-02-10T07:37:58.154083+0000","last_unstale":"2024-02-10T07:38:37.844658+0000","last_undegraded":"2024-02-10T07:38:37.844658+0000","last_fullsized":"2024-02-10T07:38:37.844658+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:18:34.317864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834388+0000","last_change":"2024-02-10T07:38:00.767983+0000","last_active":"2024-02-10T07:38:37.834388+0000","last_peered":"2024-02-10T07:38:37.834388+0000","last_clean":"2024-02-10T07:38:37.834388+0000","last_became_active":"2024-02-10T07:38:00.767609+0000","last_became_peered":"2024-02-10T07:38:00.767609+0000","last_unstale":"2024-02-10T07:38:37.834388+0000","last_undegraded":"2024-02-10T07:38:37.834388+0000","last_fullsized":"2024-02-10T07:38:37.834388+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:36:44.429584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210793+0000","last_change":"2024-02-10T07:38:02.185099+0000","last_active":"2024-02-10T07:39:03.210793+0000","last_peered":"2024-02-10T07:39:03.210793+0000","last_clean":"2024-02-10T07:39:03.210793+0000","last_became_active":"2024-02-10T07:38:02.184859+0000","last_became_peered":"2024-02-10T07:38:02.184859+0000","last_unstale":"2024-02-10T07:39:03.210793+0000","last_undegraded":"2024-02-10T07:39:03.210793+0000","last_fullsized":"2024-02-10T07:39:03.210793+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:24:27.874093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848886+0000","last_change":"2024-02-10T07:38:04.194119+0000","last_active":"2024-02-10T07:38:37.848886+0000","last_peered":"2024-02-10T07:38:37.848886+0000","last_clean":"2024-02-10T07:38:37.848886+0000","last_became_active":"2024-02-10T07:38:04.193794+0000","last_became_peered":"2024-02-10T07:38:04.193794+0000","last_unstale":"2024-02-10T07:38:37.848886+0000","last_undegraded":"2024-02-10T07:38:37.848886+0000","last_fullsized":"2024-02-10T07:38:37.848886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:59:38.728525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844561+0000","last_change":"2024-02-10T07:37:58.154699+0000","last_active":"2024-02-10T07:38:37.844561+0000","last_peered":"2024-02-10T07:38:37.844561+0000","last_clean":"2024-02-10T07:38:37.844561+0000","last_became_active":"2024-02-10T07:37:58.154367+0000","last_became_peered":"2024-02-10T07:37:58.154367+0000","last_unstale":"2024-02-10T07:38:37.844561+0000","last_undegraded":"2024-02-10T07:38:37.844561+0000","last_fullsized":"2024-02-10T07:38:37.844561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:17:52.744417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"69'12","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748001+0000","last_change":"2024-02-10T07:38:00.169422+0000","last_active":"2024-02-10T07:38:37.748001+0000","last_peered":"2024-02-10T07:38:37.748001+0000","last_clean":"2024-02-10T07:38:37.748001+0000","last_became_active":"2024-02-10T07:38:00.169111+0000","last_became_peered":"2024-02-10T07:38:00.169111+0000","last_unstale":"2024-02-10T07:38:37.748001+0000","last_undegraded":"2024-02-10T07:38:37.748001+0000","last_fullsized":"2024-02-10T07:38:37.748001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:12:36.038623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.744800+0000","last_change":"2024-02-10T07:38:02.184797+0000","last_active":"2024-02-10T07:38:37.744800+0000","last_peered":"2024-02-10T07:38:37.744800+0000","last_clean":"2024-02-10T07:38:37.744800+0000","last_became_active":"2024-02-10T07:38:02.184490+0000","last_became_peered":"2024-02-10T07:38:02.184490+0000","last_unstale":"2024-02-10T07:38:37.744800+0000","last_undegraded":"2024-02-10T07:38:37.744800+0000","last_fullsized":"2024-02-10T07:38:37.744800+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:38:09.240803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.843866+0000","last_change":"2024-02-10T07:38:04.197954+0000","last_active":"2024-02-10T07:38:37.843866+0000","last_peered":"2024-02-10T07:38:37.843866+0000","last_clean":"2024-02-10T07:38:37.843866+0000","last_became_active":"2024-02-10T07:38:04.197694+0000","last_became_peered":"2024-02-10T07:38:04.197694+0000","last_unstale":"2024-02-10T07:38:37.843866+0000","last_undegraded":"2024-02-10T07:38:37.843866+0000","last_fullsized":"2024-02-10T07:38:37.843866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:21:52.258618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.744946+0000","last_change":"2024-02-10T07:38:02.186168+0000","last_active":"2024-02-10T07:38:37.744946+0000","last_peered":"2024-02-10T07:38:37.744946+0000","last_clean":"2024-02-10T07:38:37.744946+0000","last_became_active":"2024-02-10T07:38:02.185819+0000","last_became_peered":"2024-02-10T07:38:02.185819+0000","last_unstale":"2024-02-10T07:38:37.744946+0000","last_undegraded":"2024-02-10T07:38:37.744946+0000","last_fullsized":"2024-02-10T07:38:37.744946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:08:16.039062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844784+0000","last_change":"2024-02-10T07:37:58.153698+0000","last_active":"2024-02-10T07:38:37.844784+0000","last_peered":"2024-02-10T07:38:37.844784+0000","last_clean":"2024-02-10T07:38:37.844784+0000","last_became_active":"2024-02-10T07:37:58.152990+0000","last_became_peered":"2024-02-10T07:37:58.152990+0000","last_unstale":"2024-02-10T07:38:37.844784+0000","last_undegraded":"2024-02-10T07:38:37.844784+0000","last_fullsized":"2024-02-10T07:38:37.844784+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:58:23.722396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"69'19","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847824+0000","last_change":"2024-02-10T07:38:00.766017+0000","last_active":"2024-02-10T07:38:37.847824+0000","last_peered":"2024-02-10T07:38:37.847824+0000","last_clean":"2024-02-10T07:38:37.847824+0000","last_became_active":"2024-02-10T07:38:00.765596+0000","last_became_peered":"2024-02-10T07:38:00.765596+0000","last_unstale":"2024-02-10T07:38:37.847824+0000","last_undegraded":"2024-02-10T07:38:37.847824+0000","last_fullsized":"2024-02-10T07:38:37.847824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:21:07.177967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847964+0000","last_change":"2024-02-10T07:38:04.199641+0000","last_active":"2024-02-10T07:38:37.847964+0000","last_peered":"2024-02-10T07:38:37.847964+0000","last_clean":"2024-02-10T07:38:37.847964+0000","last_became_active":"2024-02-10T07:38:04.199274+0000","last_became_peered":"2024-02-10T07:38:04.199274+0000","last_unstale":"2024-02-10T07:38:37.847964+0000","last_undegraded":"2024-02-10T07:38:37.847964+0000","last_fullsized":"2024-02-10T07:38:37.847964+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:34:12.633850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745319+0000","last_change":"2024-02-10T07:38:02.185126+0000","last_active":"2024-02-10T07:38:37.745319+0000","last_peered":"2024-02-10T07:38:37.745319+0000","last_clean":"2024-02-10T07:38:37.745319+0000","last_became_active":"2024-02-10T07:38:02.184661+0000","last_became_peered":"2024-02-10T07:38:02.184661+0000","last_unstale":"2024-02-10T07:38:37.745319+0000","last_undegraded":"2024-02-10T07:38:37.745319+0000","last_fullsized":"2024-02-10T07:38:37.745319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:16:33.565149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834804+0000","last_change":"2024-02-10T07:37:58.148204+0000","last_active":"2024-02-10T07:38:37.834804+0000","last_peered":"2024-02-10T07:38:37.834804+0000","last_clean":"2024-02-10T07:38:37.834804+0000","last_became_active":"2024-02-10T07:37:58.147899+0000","last_became_peered":"2024-02-10T07:37:58.147899+0000","last_unstale":"2024-02-10T07:38:37.834804+0000","last_undegraded":"2024-02-10T07:38:37.834804+0000","last_fullsized":"2024-02-10T07:38:37.834804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:14:58.792279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850587+0000","last_change":"2024-02-10T07:38:00.170125+0000","last_active":"2024-02-10T07:38:37.850587+0000","last_peered":"2024-02-10T07:38:37.850587+0000","last_clean":"2024-02-10T07:38:37.850587+0000","last_became_active":"2024-02-10T07:38:00.169801+0000","last_became_peered":"2024-02-10T07:38:00.169801+0000","last_unstale":"2024-02-10T07:38:37.850587+0000","last_undegraded":"2024-02-10T07:38:37.850587+0000","last_fullsized":"2024-02-10T07:38:37.850587+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:28:09.882257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850908+0000","last_change":"2024-02-10T07:38:04.194063+0000","last_active":"2024-02-10T07:38:37.850908+0000","last_peered":"2024-02-10T07:38:37.850908+0000","last_clean":"2024-02-10T07:38:37.850908+0000","last_became_active":"2024-02-10T07:38:04.193683+0000","last_became_peered":"2024-02-10T07:38:04.193683+0000","last_unstale":"2024-02-10T07:38:37.850908+0000","last_undegraded":"2024-02-10T07:38:37.850908+0000","last_fullsized":"2024-02-10T07:38:37.850908+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:10:59.476612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"65'2","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834438+0000","last_change":"2024-02-10T07:38:01.170203+0000","last_active":"2024-02-10T07:38:37.834438+0000","last_peered":"2024-02-10T07:38:37.834438+0000","last_clean":"2024-02-10T07:38:37.834438+0000","last_became_active":"2024-02-10T07:37:57.144810+0000","last_became_peered":"2024-02-10T07:37:57.144810+0000","last_unstale":"2024-02-10T07:38:37.834438+0000","last_undegraded":"2024-02-10T07:38:37.834438+0000","last_fullsized":"2024-02-10T07:38:37.834438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_clean_scrub_stamp":"2024-02-10T07:37:56.135799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:19:56.935939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017851709999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748561+0000","last_change":"2024-02-10T07:37:58.159097+0000","last_active":"2024-02-10T07:38:37.748561+0000","last_peered":"2024-02-10T07:38:37.748561+0000","last_clean":"2024-02-10T07:38:37.748561+0000","last_became_active":"2024-02-10T07:37:58.158404+0000","last_became_peered":"2024-02-10T07:37:58.158404+0000","last_unstale":"2024-02-10T07:38:37.748561+0000","last_undegraded":"2024-02-10T07:38:37.748561+0000","last_fullsized":"2024-02-10T07:38:37.748561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:55.223644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"69'28","reported_seq":92,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210313+0000","last_change":"2024-02-10T07:38:00.173241+0000","last_active":"2024-02-10T07:39:03.210313+0000","last_peered":"2024-02-10T07:39:03.210313+0000","last_clean":"2024-02-10T07:39:03.210313+0000","last_became_active":"2024-02-10T07:38:00.172634+0000","last_became_peered":"2024-02-10T07:38:00.172634+0000","last_unstale":"2024-02-10T07:39:03.210313+0000","last_undegraded":"2024-02-10T07:39:03.210313+0000","last_fullsized":"2024-02-10T07:39:03.210313+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:14:10.506150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848674+0000","last_change":"2024-02-10T07:38:02.191300+0000","last_active":"2024-02-10T07:38:37.848674+0000","last_peered":"2024-02-10T07:38:37.848674+0000","last_clean":"2024-02-10T07:38:37.848674+0000","last_became_active":"2024-02-10T07:38:02.190972+0000","last_became_peered":"2024-02-10T07:38:02.190972+0000","last_unstale":"2024-02-10T07:38:37.848674+0000","last_undegraded":"2024-02-10T07:38:37.848674+0000","last_fullsized":"2024-02-10T07:38:37.848674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:55:16.795143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844933+0000","last_change":"2024-02-10T07:38:04.561735+0000","last_active":"2024-02-10T07:38:37.844933+0000","last_peered":"2024-02-10T07:38:37.844933+0000","last_clean":"2024-02-10T07:38:37.844933+0000","last_became_active":"2024-02-10T07:38:04.561417+0000","last_became_peered":"2024-02-10T07:38:04.561417+0000","last_unstale":"2024-02-10T07:38:37.844933+0000","last_undegraded":"2024-02-10T07:38:37.844933+0000","last_fullsized":"2024-02-10T07:38:37.844933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:55:04.134680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849450+0000","last_change":"2024-02-10T07:37:58.151796+0000","last_active":"2024-02-10T07:38:37.849450+0000","last_peered":"2024-02-10T07:38:37.849450+0000","last_clean":"2024-02-10T07:38:37.849450+0000","last_became_active":"2024-02-10T07:37:58.151501+0000","last_became_peered":"2024-02-10T07:37:58.151501+0000","last_unstale":"2024-02-10T07:38:37.849450+0000","last_undegraded":"2024-02-10T07:38:37.849450+0000","last_fullsized":"2024-02-10T07:38:37.849450+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:13:28.280284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.744807+0000","last_change":"2024-02-10T07:38:01.167119+0000","last_active":"2024-02-10T07:38:37.744807+0000","last_peered":"2024-02-10T07:38:37.744807+0000","last_clean":"2024-02-10T07:38:37.744807+0000","last_became_active":"2024-02-10T07:37:57.145944+0000","last_became_peered":"2024-02-10T07:37:57.145944+0000","last_unstale":"2024-02-10T07:38:37.744807+0000","last_undegraded":"2024-02-10T07:38:37.744807+0000","last_fullsized":"2024-02-10T07:38:37.744807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_clean_scrub_stamp":"2024-02-10T07:37:56.135799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:53:49.849478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050838299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"69'13","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.746819+0000","last_change":"2024-02-10T07:38:28.792025+0000","last_active":"2024-02-10T07:38:37.746819+0000","last_peered":"2024-02-10T07:38:37.746819+0000","last_clean":"2024-02-10T07:38:37.746819+0000","last_became_active":"2024-02-10T07:38:28.776679+0000","last_became_peered":"2024-02-10T07:38:28.776679+0000","last_unstale":"2024-02-10T07:38:37.746819+0000","last_undegraded":"2024-02-10T07:38:37.746819+0000","last_fullsized":"2024-02-10T07:38:37.746819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:00:43.901323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745918+0000","last_change":"2024-02-10T07:38:02.184270+0000","last_active":"2024-02-10T07:38:37.745918+0000","last_peered":"2024-02-10T07:38:37.745918+0000","last_clean":"2024-02-10T07:38:37.745918+0000","last_became_active":"2024-02-10T07:38:02.183930+0000","last_became_peered":"2024-02-10T07:38:02.183930+0000","last_unstale":"2024-02-10T07:38:37.745918+0000","last_undegraded":"2024-02-10T07:38:37.745918+0000","last_fullsized":"2024-02-10T07:38:37.745918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:42:04.396161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834724+0000","last_change":"2024-02-10T07:38:04.193266+0000","last_active":"2024-02-10T07:38:37.834724+0000","last_peered":"2024-02-10T07:38:37.834724+0000","last_clean":"2024-02-10T07:38:37.834724+0000","last_became_active":"2024-02-10T07:38:04.192951+0000","last_became_peered":"2024-02-10T07:38:04.192951+0000","last_unstale":"2024-02-10T07:38:37.834724+0000","last_undegraded":"2024-02-10T07:38:37.834724+0000","last_fullsized":"2024-02-10T07:38:37.834724+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:54:15.160943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":278,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:39.832820+0000","last_change":"2024-02-10T07:37:47.705354+0000","last_active":"2024-02-10T07:38:39.832820+0000","last_peered":"2024-02-10T07:38:39.832820+0000","last_clean":"2024-02-10T07:38:39.832820+0000","last_became_active":"2024-02-10T07:37:47.704903+0000","last_became_peered":"2024-02-10T07:37:47.704903+0000","last_unstale":"2024-02-10T07:38:39.832820+0000","last_undegraded":"2024-02-10T07:38:39.832820+0000","last_fullsized":"2024-02-10T07:38:39.832820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:35:41.758084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:35:41.758084+0000","last_clean_scrub_stamp":"2024-02-10T07:35:41.758084+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:54:25.399467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848639+0000","last_change":"2024-02-10T07:37:58.151789+0000","last_active":"2024-02-10T07:38:37.848639+0000","last_peered":"2024-02-10T07:38:37.848639+0000","last_clean":"2024-02-10T07:38:37.848639+0000","last_became_active":"2024-02-10T07:37:58.151497+0000","last_became_peered":"2024-02-10T07:37:58.151497+0000","last_unstale":"2024-02-10T07:38:37.848639+0000","last_undegraded":"2024-02-10T07:38:37.848639+0000","last_fullsized":"2024-02-10T07:38:37.848639+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:33:00.081823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"69'16","reported_seq":66,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210811+0000","last_change":"2024-02-10T07:38:00.767903+0000","last_active":"2024-02-10T07:39:03.210811+0000","last_peered":"2024-02-10T07:39:03.210811+0000","last_clean":"2024-02-10T07:39:03.210811+0000","last_became_active":"2024-02-10T07:38:00.767587+0000","last_became_peered":"2024-02-10T07:38:00.767587+0000","last_unstale":"2024-02-10T07:39:03.210811+0000","last_undegraded":"2024-02-10T07:39:03.210811+0000","last_fullsized":"2024-02-10T07:39:03.210811+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:31:33.266260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844488+0000","last_change":"2024-02-10T07:38:02.188884+0000","last_active":"2024-02-10T07:38:37.844488+0000","last_peered":"2024-02-10T07:38:37.844488+0000","last_clean":"2024-02-10T07:38:37.844488+0000","last_became_active":"2024-02-10T07:38:02.188374+0000","last_became_peered":"2024-02-10T07:38:02.188374+0000","last_unstale":"2024-02-10T07:38:37.844488+0000","last_undegraded":"2024-02-10T07:38:37.844488+0000","last_fullsized":"2024-02-10T07:38:37.844488+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:28:53.296511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848833+0000","last_change":"2024-02-10T07:38:04.191260+0000","last_active":"2024-02-10T07:38:37.848833+0000","last_peered":"2024-02-10T07:38:37.848833+0000","last_clean":"2024-02-10T07:38:37.848833+0000","last_became_active":"2024-02-10T07:38:04.190966+0000","last_became_peered":"2024-02-10T07:38:04.190966+0000","last_unstale":"2024-02-10T07:38:37.848833+0000","last_undegraded":"2024-02-10T07:38:37.848833+0000","last_fullsized":"2024-02-10T07:38:37.848833+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:33:25.377619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848164+0000","last_change":"2024-02-10T07:37:58.157363+0000","last_active":"2024-02-10T07:38:37.848164+0000","last_peered":"2024-02-10T07:38:37.848164+0000","last_clean":"2024-02-10T07:38:37.848164+0000","last_became_active":"2024-02-10T07:37:58.157042+0000","last_became_peered":"2024-02-10T07:37:58.157042+0000","last_unstale":"2024-02-10T07:38:37.848164+0000","last_undegraded":"2024-02-10T07:38:37.848164+0000","last_fullsized":"2024-02-10T07:38:37.848164+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:18:17.067317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"69'5","reported_seq":108,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:07.015490+0000","last_change":"2024-02-10T07:38:01.169836+0000","last_active":"2024-02-10T07:39:07.015490+0000","last_peered":"2024-02-10T07:39:07.015490+0000","last_clean":"2024-02-10T07:39:07.015490+0000","last_became_active":"2024-02-10T07:37:57.147550+0000","last_became_peered":"2024-02-10T07:37:57.147550+0000","last_unstale":"2024-02-10T07:39:07.015490+0000","last_undegraded":"2024-02-10T07:39:07.015490+0000","last_fullsized":"2024-02-10T07:39:07.015490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_clean_scrub_stamp":"2024-02-10T07:37:56.135799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:45:42.849143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090214500000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"69'12","reported_seq":47,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849359+0000","last_change":"2024-02-10T07:38:00.166353+0000","last_active":"2024-02-10T07:38:37.849359+0000","last_peered":"2024-02-10T07:38:37.849359+0000","last_clean":"2024-02-10T07:38:37.849359+0000","last_became_active":"2024-02-10T07:38:00.165935+0000","last_became_peered":"2024-02-10T07:38:00.165935+0000","last_unstale":"2024-02-10T07:38:37.849359+0000","last_undegraded":"2024-02-10T07:38:37.849359+0000","last_fullsized":"2024-02-10T07:38:37.849359+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:36:42.497334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.835018+0000","last_change":"2024-02-10T07:38:02.188399+0000","last_active":"2024-02-10T07:38:37.835018+0000","last_peered":"2024-02-10T07:38:37.835018+0000","last_clean":"2024-02-10T07:38:37.835018+0000","last_became_active":"2024-02-10T07:38:02.188087+0000","last_became_peered":"2024-02-10T07:38:02.188087+0000","last_unstale":"2024-02-10T07:38:37.835018+0000","last_undegraded":"2024-02-10T07:38:37.835018+0000","last_fullsized":"2024-02-10T07:38:37.835018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:39:28.135958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849012+0000","last_change":"2024-02-10T07:38:04.561979+0000","last_active":"2024-02-10T07:38:37.849012+0000","last_peered":"2024-02-10T07:38:37.849012+0000","last_clean":"2024-02-10T07:38:37.849012+0000","last_became_active":"2024-02-10T07:38:04.561422+0000","last_became_peered":"2024-02-10T07:38:04.561422+0000","last_unstale":"2024-02-10T07:38:37.849012+0000","last_undegraded":"2024-02-10T07:38:37.849012+0000","last_fullsized":"2024-02-10T07:38:37.849012+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:08:10.746152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849945+0000","last_change":"2024-02-10T07:37:58.156376+0000","last_active":"2024-02-10T07:38:37.849945+0000","last_peered":"2024-02-10T07:38:37.849945+0000","last_clean":"2024-02-10T07:38:37.849945+0000","last_became_active":"2024-02-10T07:37:58.156096+0000","last_became_peered":"2024-02-10T07:37:58.156096+0000","last_unstale":"2024-02-10T07:38:37.849945+0000","last_undegraded":"2024-02-10T07:38:37.849945+0000","last_fullsized":"2024-02-10T07:38:37.849945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:20:52.614395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"69'19","reported_seq":65,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850338+0000","last_change":"2024-02-10T07:38:00.765853+0000","last_active":"2024-02-10T07:38:37.850338+0000","last_peered":"2024-02-10T07:38:37.850338+0000","last_clean":"2024-02-10T07:38:37.850338+0000","last_became_active":"2024-02-10T07:38:00.765543+0000","last_became_peered":"2024-02-10T07:38:00.765543+0000","last_unstale":"2024-02-10T07:38:37.850338+0000","last_undegraded":"2024-02-10T07:38:37.850338+0000","last_fullsized":"2024-02-10T07:38:37.850338+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:35:14.838796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847870+0000","last_change":"2024-02-10T07:38:02.184373+0000","last_active":"2024-02-10T07:38:37.847870+0000","last_peered":"2024-02-10T07:38:37.847870+0000","last_clean":"2024-02-10T07:38:37.847870+0000","last_became_active":"2024-02-10T07:38:02.183764+0000","last_became_peered":"2024-02-10T07:38:02.183764+0000","last_unstale":"2024-02-10T07:38:37.847870+0000","last_undegraded":"2024-02-10T07:38:37.847870+0000","last_fullsized":"2024-02-10T07:38:37.847870+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:03:44.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847821+0000","last_change":"2024-02-10T07:38:04.199423+0000","last_active":"2024-02-10T07:38:37.847821+0000","last_peered":"2024-02-10T07:38:37.847821+0000","last_clean":"2024-02-10T07:38:37.847821+0000","last_became_active":"2024-02-10T07:38:04.199058+0000","last_became_peered":"2024-02-10T07:38:04.199058+0000","last_unstale":"2024-02-10T07:38:37.847821+0000","last_undegraded":"2024-02-10T07:38:37.847821+0000","last_fullsized":"2024-02-10T07:38:37.847821+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:28:11.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834994+0000","last_change":"2024-02-10T07:37:58.156606+0000","last_active":"2024-02-10T07:38:37.834994+0000","last_peered":"2024-02-10T07:38:37.834994+0000","last_clean":"2024-02-10T07:38:37.834994+0000","last_became_active":"2024-02-10T07:37:58.156297+0000","last_became_peered":"2024-02-10T07:37:58.156297+0000","last_unstale":"2024-02-10T07:38:37.834994+0000","last_undegraded":"2024-02-10T07:38:37.834994+0000","last_fullsized":"2024-02-10T07:38:37.834994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:21.511715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"69'10","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745081+0000","last_change":"2024-02-10T07:38:28.781809+0000","last_active":"2024-02-10T07:38:37.745081+0000","last_peered":"2024-02-10T07:38:37.745081+0000","last_clean":"2024-02-10T07:38:37.745081+0000","last_became_active":"2024-02-10T07:38:28.776353+0000","last_became_peered":"2024-02-10T07:38:28.776353+0000","last_unstale":"2024-02-10T07:38:37.745081+0000","last_undegraded":"2024-02-10T07:38:37.745081+0000","last_fullsized":"2024-02-10T07:38:37.745081+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:35:00.739241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210352+0000","last_change":"2024-02-10T07:38:02.184061+0000","last_active":"2024-02-10T07:39:03.210352+0000","last_peered":"2024-02-10T07:39:03.210352+0000","last_clean":"2024-02-10T07:39:03.210352+0000","last_became_active":"2024-02-10T07:38:02.183267+0000","last_became_peered":"2024-02-10T07:38:02.183267+0000","last_unstale":"2024-02-10T07:39:03.210352+0000","last_undegraded":"2024-02-10T07:39:03.210352+0000","last_fullsized":"2024-02-10T07:39:03.210352+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:16:15.305483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844383+0000","last_change":"2024-02-10T07:38:04.563780+0000","last_active":"2024-02-10T07:38:37.844383+0000","last_peered":"2024-02-10T07:38:37.844383+0000","last_clean":"2024-02-10T07:38:37.844383+0000","last_became_active":"2024-02-10T07:38:04.563497+0000","last_became_peered":"2024-02-10T07:38:04.563497+0000","last_unstale":"2024-02-10T07:38:37.844383+0000","last_undegraded":"2024-02-10T07:38:37.844383+0000","last_fullsized":"2024-02-10T07:38:37.844383+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:53:01.554104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849492+0000","last_change":"2024-02-10T07:37:58.156387+0000","last_active":"2024-02-10T07:38:37.849492+0000","last_peered":"2024-02-10T07:38:37.849492+0000","last_clean":"2024-02-10T07:38:37.849492+0000","last_became_active":"2024-02-10T07:37:58.156090+0000","last_became_peered":"2024-02-10T07:37:58.156090+0000","last_unstale":"2024-02-10T07:38:37.849492+0000","last_undegraded":"2024-02-10T07:38:37.849492+0000","last_fullsized":"2024-02-10T07:38:37.849492+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:42:25.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"69'14","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747779+0000","last_change":"2024-02-10T07:38:00.160755+0000","last_active":"2024-02-10T07:38:37.747779+0000","last_peered":"2024-02-10T07:38:37.747779+0000","last_clean":"2024-02-10T07:38:37.747779+0000","last_became_active":"2024-02-10T07:38:00.160279+0000","last_became_peered":"2024-02-10T07:38:00.160279+0000","last_unstale":"2024-02-10T07:38:37.747779+0000","last_undegraded":"2024-02-10T07:38:37.747779+0000","last_fullsized":"2024-02-10T07:38:37.747779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:37:03.981649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210302+0000","last_change":"2024-02-10T07:38:02.186969+0000","last_active":"2024-02-10T07:39:03.210302+0000","last_peered":"2024-02-10T07:39:03.210302+0000","last_clean":"2024-02-10T07:39:03.210302+0000","last_became_active":"2024-02-10T07:38:02.186668+0000","last_became_peered":"2024-02-10T07:38:02.186668+0000","last_unstale":"2024-02-10T07:39:03.210302+0000","last_undegraded":"2024-02-10T07:39:03.210302+0000","last_fullsized":"2024-02-10T07:39:03.210302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:33:09.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747966+0000","last_change":"2024-02-10T07:38:04.193946+0000","last_active":"2024-02-10T07:38:37.747966+0000","last_peered":"2024-02-10T07:38:37.747966+0000","last_clean":"2024-02-10T07:38:37.747966+0000","last_became_active":"2024-02-10T07:38:04.193690+0000","last_became_peered":"2024-02-10T07:38:04.193690+0000","last_unstale":"2024-02-10T07:38:37.747966+0000","last_undegraded":"2024-02-10T07:38:37.747966+0000","last_fullsized":"2024-02-10T07:38:37.747966+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:45:47.474491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851530+0000","last_change":"2024-02-10T07:37:58.152566+0000","last_active":"2024-02-10T07:38:37.851530+0000","last_peered":"2024-02-10T07:38:37.851530+0000","last_clean":"2024-02-10T07:38:37.851530+0000","last_became_active":"2024-02-10T07:37:58.152282+0000","last_became_peered":"2024-02-10T07:37:58.152282+0000","last_unstale":"2024-02-10T07:38:37.851530+0000","last_undegraded":"2024-02-10T07:38:37.851530+0000","last_fullsized":"2024-02-10T07:38:37.851530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:49:59.465503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"69'18","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848169+0000","last_change":"2024-02-10T07:38:00.765936+0000","last_active":"2024-02-10T07:38:37.848169+0000","last_peered":"2024-02-10T07:38:37.848169+0000","last_clean":"2024-02-10T07:38:37.848169+0000","last_became_active":"2024-02-10T07:38:00.765527+0000","last_became_peered":"2024-02-10T07:38:00.765527+0000","last_unstale":"2024-02-10T07:38:37.848169+0000","last_undegraded":"2024-02-10T07:38:37.848169+0000","last_fullsized":"2024-02-10T07:38:37.848169+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:37:57.750880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.749028+0000","last_change":"2024-02-10T07:38:02.186166+0000","last_active":"2024-02-10T07:38:37.749028+0000","last_peered":"2024-02-10T07:38:37.749028+0000","last_clean":"2024-02-10T07:38:37.749028+0000","last_became_active":"2024-02-10T07:38:02.185756+0000","last_became_peered":"2024-02-10T07:38:02.185756+0000","last_unstale":"2024-02-10T07:38:37.749028+0000","last_undegraded":"2024-02-10T07:38:37.749028+0000","last_fullsized":"2024-02-10T07:38:37.749028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:09.439249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834707+0000","last_change":"2024-02-10T07:38:04.190094+0000","last_active":"2024-02-10T07:38:37.834707+0000","last_peered":"2024-02-10T07:38:37.834707+0000","last_clean":"2024-02-10T07:38:37.834707+0000","last_became_active":"2024-02-10T07:38:04.189824+0000","last_became_peered":"2024-02-10T07:38:04.189824+0000","last_unstale":"2024-02-10T07:38:37.834707+0000","last_undegraded":"2024-02-10T07:38:37.834707+0000","last_fullsized":"2024-02-10T07:38:37.834707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:40:06.349827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":51,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210786+0000","last_change":"2024-02-10T07:38:02.184785+0000","last_active":"2024-02-10T07:39:03.210786+0000","last_peered":"2024-02-10T07:39:03.210786+0000","last_clean":"2024-02-10T07:39:03.210786+0000","last_became_active":"2024-02-10T07:38:02.183131+0000","last_became_peered":"2024-02-10T07:38:02.183131+0000","last_unstale":"2024-02-10T07:39:03.210786+0000","last_undegraded":"2024-02-10T07:39:03.210786+0000","last_fullsized":"2024-02-10T07:39:03.210786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:39:22.937185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848084+0000","last_change":"2024-02-10T07:37:58.153227+0000","last_active":"2024-02-10T07:38:37.848084+0000","last_peered":"2024-02-10T07:38:37.848084+0000","last_clean":"2024-02-10T07:38:37.848084+0000","last_became_active":"2024-02-10T07:37:58.152937+0000","last_became_peered":"2024-02-10T07:37:58.152937+0000","last_unstale":"2024-02-10T07:38:37.848084+0000","last_undegraded":"2024-02-10T07:38:37.848084+0000","last_fullsized":"2024-02-10T07:38:37.848084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:55:00.998754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848566+0000","last_change":"2024-02-10T07:38:00.170311+0000","last_active":"2024-02-10T07:38:37.848566+0000","last_peered":"2024-02-10T07:38:37.848566+0000","last_clean":"2024-02-10T07:38:37.848566+0000","last_became_active":"2024-02-10T07:38:00.169727+0000","last_became_peered":"2024-02-10T07:38:00.169727+0000","last_unstale":"2024-02-10T07:38:37.848566+0000","last_undegraded":"2024-02-10T07:38:37.848566+0000","last_fullsized":"2024-02-10T07:38:37.848566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:58:13.341066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850101+0000","last_change":"2024-02-10T07:38:04.562786+0000","last_active":"2024-02-10T07:38:37.850101+0000","last_peered":"2024-02-10T07:38:37.850101+0000","last_clean":"2024-02-10T07:38:37.850101+0000","last_became_active":"2024-02-10T07:38:04.561360+0000","last_became_peered":"2024-02-10T07:38:04.561360+0000","last_unstale":"2024-02-10T07:38:37.850101+0000","last_undegraded":"2024-02-10T07:38:37.850101+0000","last_fullsized":"2024-02-10T07:38:37.850101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:13.927336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834918+0000","last_change":"2024-02-10T07:38:02.184847+0000","last_active":"2024-02-10T07:38:37.834918+0000","last_peered":"2024-02-10T07:38:37.834918+0000","last_clean":"2024-02-10T07:38:37.834918+0000","last_became_active":"2024-02-10T07:38:02.184558+0000","last_became_peered":"2024-02-10T07:38:02.184558+0000","last_unstale":"2024-02-10T07:38:37.834918+0000","last_undegraded":"2024-02-10T07:38:37.834918+0000","last_fullsized":"2024-02-10T07:38:37.834918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:34:57.087286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747625+0000","last_change":"2024-02-10T07:37:58.157306+0000","last_active":"2024-02-10T07:38:37.747625+0000","last_peered":"2024-02-10T07:38:37.747625+0000","last_clean":"2024-02-10T07:38:37.747625+0000","last_became_active":"2024-02-10T07:37:58.156985+0000","last_became_peered":"2024-02-10T07:37:58.156985+0000","last_unstale":"2024-02-10T07:38:37.747625+0000","last_undegraded":"2024-02-10T07:38:37.747625+0000","last_fullsized":"2024-02-10T07:38:37.747625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:21:32.639102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.750137+0000","last_change":"2024-02-10T07:38:00.162568+0000","last_active":"2024-02-10T07:38:37.750137+0000","last_peered":"2024-02-10T07:38:37.750137+0000","last_clean":"2024-02-10T07:38:37.750137+0000","last_became_active":"2024-02-10T07:38:00.162259+0000","last_became_peered":"2024-02-10T07:38:00.162259+0000","last_unstale":"2024-02-10T07:38:37.750137+0000","last_undegraded":"2024-02-10T07:38:37.750137+0000","last_fullsized":"2024-02-10T07:38:37.750137+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:52:51.018898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747024+0000","last_change":"2024-02-10T07:38:04.200985+0000","last_active":"2024-02-10T07:38:37.747024+0000","last_peered":"2024-02-10T07:38:37.747024+0000","last_clean":"2024-02-10T07:38:37.747024+0000","last_became_active":"2024-02-10T07:38:04.200613+0000","last_became_peered":"2024-02-10T07:38:04.200613+0000","last_unstale":"2024-02-10T07:38:37.747024+0000","last_undegraded":"2024-02-10T07:38:37.747024+0000","last_fullsized":"2024-02-10T07:38:37.747024+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:43:21.024175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850158+0000","last_change":"2024-02-10T07:38:02.182150+0000","last_active":"2024-02-10T07:38:37.850158+0000","last_peered":"2024-02-10T07:38:37.850158+0000","last_clean":"2024-02-10T07:38:37.850158+0000","last_became_active":"2024-02-10T07:38:02.181811+0000","last_became_peered":"2024-02-10T07:38:02.181811+0000","last_unstale":"2024-02-10T07:38:37.850158+0000","last_undegraded":"2024-02-10T07:38:37.850158+0000","last_fullsized":"2024-02-10T07:38:37.850158+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:05:07.917637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847525+0000","last_change":"2024-02-10T07:37:58.157114+0000","last_active":"2024-02-10T07:38:37.847525+0000","last_peered":"2024-02-10T07:38:37.847525+0000","last_clean":"2024-02-10T07:38:37.847525+0000","last_became_active":"2024-02-10T07:37:58.156848+0000","last_became_peered":"2024-02-10T07:37:58.156848+0000","last_unstale":"2024-02-10T07:38:37.847525+0000","last_undegraded":"2024-02-10T07:38:37.847525+0000","last_fullsized":"2024-02-10T07:38:37.847525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:33:39.433851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"69'17","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.749712+0000","last_change":"2024-02-10T07:38:00.168389+0000","last_active":"2024-02-10T07:38:37.749712+0000","last_peered":"2024-02-10T07:38:37.749712+0000","last_clean":"2024-02-10T07:38:37.749712+0000","last_became_active":"2024-02-10T07:38:00.168111+0000","last_became_peered":"2024-02-10T07:38:00.168111+0000","last_unstale":"2024-02-10T07:38:37.749712+0000","last_undegraded":"2024-02-10T07:38:37.749712+0000","last_fullsized":"2024-02-10T07:38:37.749712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:08:01.001880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.750431+0000","last_change":"2024-02-10T07:38:04.190427+0000","last_active":"2024-02-10T07:38:37.750431+0000","last_peered":"2024-02-10T07:38:37.750431+0000","last_clean":"2024-02-10T07:38:37.750431+0000","last_became_active":"2024-02-10T07:38:04.190138+0000","last_became_peered":"2024-02-10T07:38:04.190138+0000","last_unstale":"2024-02-10T07:38:37.750431+0000","last_undegraded":"2024-02-10T07:38:37.750431+0000","last_fullsized":"2024-02-10T07:38:37.750431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:40:22.147921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:03.210331+0000","last_change":"2024-02-10T07:38:02.184972+0000","last_active":"2024-02-10T07:39:03.210331+0000","last_peered":"2024-02-10T07:39:03.210331+0000","last_clean":"2024-02-10T07:39:03.210331+0000","last_became_active":"2024-02-10T07:38:02.183251+0000","last_became_peered":"2024-02-10T07:38:02.183251+0000","last_unstale":"2024-02-10T07:39:03.210331+0000","last_undegraded":"2024-02-10T07:39:03.210331+0000","last_fullsized":"2024-02-10T07:39:03.210331+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:32:49.562659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849142+0000","last_change":"2024-02-10T07:37:58.154971+0000","last_active":"2024-02-10T07:38:37.849142+0000","last_peered":"2024-02-10T07:38:37.849142+0000","last_clean":"2024-02-10T07:38:37.849142+0000","last_became_active":"2024-02-10T07:37:58.154664+0000","last_became_peered":"2024-02-10T07:37:58.154664+0000","last_unstale":"2024-02-10T07:38:37.849142+0000","last_undegraded":"2024-02-10T07:38:37.849142+0000","last_fullsized":"2024-02-10T07:38:37.849142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:08:09.230585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748254+0000","last_change":"2024-02-10T07:38:00.164094+0000","last_active":"2024-02-10T07:38:37.748254+0000","last_peered":"2024-02-10T07:38:37.748254+0000","last_clean":"2024-02-10T07:38:37.748254+0000","last_became_active":"2024-02-10T07:38:00.163805+0000","last_became_peered":"2024-02-10T07:38:00.163805+0000","last_unstale":"2024-02-10T07:38:37.748254+0000","last_undegraded":"2024-02-10T07:38:37.748254+0000","last_fullsized":"2024-02-10T07:38:37.748254+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:01:35.721568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":58,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201299,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28304,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716848,"statfs":{"total":95995035648,"available":95966052352,"internally_reserved":0,"allocated":1318912,"data_stored":874659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46999999999999997}]},{"osd":1,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.502}]},{"osd":2,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.46000000000000002}]},{"osd":4,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.442},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.35699999999999998}]},{"osd":5,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55700000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462936,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32448,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712704,"statfs":{"total":95995035648,"available":95961808896,"internally_reserved":0,"allocated":1302528,"data_stored":873621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57999999999999996}]},{"osd":1,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":12.202999999999999,"5min":12.202999999999999,"15min":12.202999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60099999999999998}]},{"osd":2,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.621}]},{"osd":3,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":12.118,"5min":12.118,"15min":12.118},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.70199999999999996}]},{"osd":4,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.75}]},{"osd":5,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.79200000000000004}]},{"osd":7,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717396,"statfs":{"total":95995035648,"available":95966613504,"internally_reserved":0,"allocated":753664,"data_stored":284585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:28 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.80900000000000005,"15min":0.99099999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.58799999999999997,"5min":16.248999999999999,"15min":16.248999999999999},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.33800000000000002}]},{"osd":2,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40999999999999998},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.625,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36099999999999999}]},{"osd":3,"last update":"Sat Feb 10 07:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.496,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.374}]},{"osd":4,"last update":"Sat Feb 10 07:38:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.503}]},{"osd":5,"last update":"Sat Feb 10 07:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75900000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63600000000000001,"15min":0.64500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.75600000000000001}]},{"osd":6,"last update":"Sat Feb 10 07:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.73499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":875674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.35599999999999998,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.193,"15min":0.193},"max":{"1min":0.58399999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.371},"min":{"1min":0.33200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.379}]},{"osd":2,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.39900000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54400000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.41099999999999998},"min":{"1min":0.29999999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.34399999999999997}]},{"osd":3,"last update":"Sat Feb 10 07:38:06 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.47099999999999997,"15min":0.47799999999999998},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56000000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.57299999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.36399999999999999}]},{"osd":4,"last update":"Sat Feb 10 07:38:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.61399999999999999}]},{"osd":5,"last update":"Sat Feb 10 07:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Feb 10 07:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.624},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65300000000000002}]},{"osd":7,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60199999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":720896,"data_stored":284192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.372,"15min":0.376},"min":{"1min":0.26600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51800000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.35199999999999998}]},{"osd":1,"last update":"Sat Feb 10 07:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.41499999999999998,"15min":0.40500000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.246},{"interface":"front","average":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.28000000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.30299999999999999}]},{"osd":3,"last update":"Sat Feb 10 07:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64500000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.371},{"interface":"front","average":{"1min":0.434,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.40200000000000002}]},{"osd":4,"last update":"Sat Feb 10 07:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.55700000000000005}]},{"osd":5,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.50700000000000001}]},{"osd":6,"last update":"Sat Feb 10 07:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57699999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149734,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717400,"statfs":{"total":95995035648,"available":95966617600,"internally_reserved":0,"allocated":749568,"data_stored":285018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":5.1920000000000002,"15min":5.1920000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.48699999999999999}]},{"osd":1,"last update":"Sat Feb 10 07:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.442,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68000000000000005,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.435},{"interface":"front","average":{"1min":0.373,"5min":0.432,"15min":0.437},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65400000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.502}]},{"osd":2,"last update":"Sat Feb 10 07:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44700000000000001,"15min":0.45700000000000002},"min":{"1min":0.28100000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.50900000000000001,"15min":0.46999999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70999999999999996,"5min":5.1470000000000002,"15min":5.1470000000000002},"last":0.41299999999999998}]},{"osd":4,"last update":"Sat Feb 10 07:38:31 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.623,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.1180000000000003,"5min":5.1180000000000003,"15min":5.1180000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.64500000000000002,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.3819999999999997,"5min":7.3819999999999997,"15min":7.3819999999999997},"last":0.78800000000000003}]},{"osd":5,"last update":"Sat Feb 10 07:38:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.077,"5min":5.077,"15min":5.077},"last":0.755}]},{"osd":6,"last update":"Sat Feb 10 07:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.2210000000000001,"5min":5.2210000000000001,"15min":5.2210000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Sat Feb 10 07:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.74299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":283845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.69299999999999995}]},{"osd":1,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":1.2529999999999999,"15min":1.3819999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":18.132999999999999,"15min":18.132999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.76600000000000001}]},{"osd":3,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":1.2829999999999999,"15min":1.411},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.64600000000000002,"5min":18.106999999999999,"15min":18.106999999999999},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.39100000000000001},"min":{"1min":0.27300000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.745}]},{"osd":6,"last update":"Sat Feb 10 07:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Feb 10 07:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.64800000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724573,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713268,"statfs":{"total":95995035648,"available":95962386432,"internally_reserved":0,"allocated":720896,"data_stored":284850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68500000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.54900000000000004}]},{"osd":1,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.497}]},{"osd":2,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73099999999999998,"5min":0.746,"15min":0.746},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.40000000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.627}]},{"osd":3,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"min":{"1min":0.309,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.59299999999999997}]},{"osd":6,"last update":"Sat Feb 10 07:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.70199999999999996}]},{"osd":7,"last update":"Sat Feb 10 07:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.61599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T07:39:14.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T07:39:14.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[29548]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:39:14.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[29548]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:39:14.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[29548]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:39:14.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[29548]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:39:14.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[29548]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[29548]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[29548]: Deploying daemon alertmanager.a on smithi070 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[39923]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[39923]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[39923]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[39923]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[39923]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[39923]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:39:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:14 smithi070 ceph-mon[39923]: Deploying daemon alertmanager.a on smithi070 2024-02-10T07:39:14.500 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:39:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:14 smithi155 ceph-mon[32688]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:39:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:14 smithi155 ceph-mon[32688]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:39:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:14 smithi155 ceph-mon[32688]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:39:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:14 smithi155 ceph-mon[32688]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:39:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:14 smithi155 ceph-mon[32688]: Updating smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:39:14.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:14 smithi155 ceph-mon[32688]: Updating smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:39:14.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:14 smithi155 ceph-mon[32688]: Deploying daemon alertmanager.a on smithi070 2024-02-10T07:39:15.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:15 smithi070 ceph-mon[29548]: from='client.24646 v1:172.21.15.70:0/1145940833' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:39:15.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:15 smithi070 ceph-mon[29548]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:15.720 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:15 smithi070 ceph-mon[39923]: from='client.24646 v1:172.21.15.70:0/1145940833' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:39:15.720 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:15 smithi070 ceph-mon[39923]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:15 smithi155 ceph-mon[32688]: from='client.24646 v1:172.21.15.70:0/1145940833' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:39:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:15 smithi155 ceph-mon[32688]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.568+0000 7f51bf9e1700 1 -- 172.21.15.70:0/1878716131 >> v1:172.21.15.70:6790/0 conn(0x7f51b81059a0 legacy=0x7f51b8105d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.568+0000 7f51bf9e1700 1 -- 172.21.15.70:0/1878716131 shutdown_connections 2024-02-10T07:39:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.568+0000 7f51bf9e1700 1 -- 172.21.15.70:0/1878716131 >> 172.21.15.70:0/1878716131 conn(0x7f51b8076e50 msgr2=0x7f51b80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.569+0000 7f51bf9e1700 1 -- 172.21.15.70:0/1878716131 shutdown_connections 2024-02-10T07:39:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.569+0000 7f51bf9e1700 1 -- 172.21.15.70:0/1878716131 wait complete. 2024-02-10T07:39:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.569+0000 7f51bf9e1700 1 Processor -- start 2024-02-10T07:39:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.569+0000 7f51bf9e1700 1 -- start start 2024-02-10T07:39:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51bf9e1700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f51b81a68f0 con 0x7f51b81059a0 2024-02-10T07:39:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51bf9e1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f51b81a7ad0 con 0x7f51b810d500 2024-02-10T07:39:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51bf9e1700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f51b81a8cd0 con 0x7f51b8109800 2024-02-10T07:39:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51bcf7c700 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f51b8109800 0x7f51b81a18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:34678/0 (socket says 172.21.15.70:34678) 2024-02-10T07:39:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51bcf7c700 1 -- 172.21.15.70:0/2026428545 learned_addr learned my addr 172.21.15.70:0/2026428545 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51bd77d700 1 --1- 172.21.15.70:0/2026428545 >> v1:172.21.15.70:6789/0 conn(0x7f51b81059a0 0x7f51b80fef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57002/0 (socket says 172.21.15.70:57002) 2024-02-10T07:39:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131321457 0 0) 0x7f51b81a8cd0 con 0x7f51b8109800 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a40035b0 con 0x7f51b8109800 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870018327 0 0) 0x7f51b81a7ad0 con 0x7f51b810d500 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b81a8cd0 con 0x7f51b810d500 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3433545055 0 0) 0x7f51a40035b0 con 0x7f51b8109800 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51b81a7ad0 con 0x7f51b8109800 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f51a8001970 con 0x7f51b8109800 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 212779482 0 0) 0x7f51b81a8cd0 con 0x7f51b810d500 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51a40035b0 con 0x7f51b810d500 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.570+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f51b0001b70 con 0x7f51b810d500 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 176848059 0 0) 0x7f51b81a7ad0 con 0x7f51b8109800 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 >> v1:172.21.15.155:6789/0 conn(0x7f51b810d500 legacy=0x7f51b81a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:16.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 >> v1:172.21.15.70:6789/0 conn(0x7f51b81059a0 legacy=0x7f51b80fef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b81a9ed0 con 0x7f51b8109800 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b81a7cc0 con 0x7f51b8109800 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b81a8180 con 0x7f51b8109800 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f51a8003b10 con 0x7f51b8109800 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.571+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f51a8005c10 con 0x7f51b8109800 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.572+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c005120 con 0x7f51b8109800 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.572+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f51a8003680 con 0x7f51b8109800 2024-02-10T07:39:16.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.573+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f51a8088b00 con 0x7f51b8109800 2024-02-10T07:39:16.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.578+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f51a8055e00 con 0x7f51b8109800 2024-02-10T07:39:16.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.792+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 --> v1:172.21.15.70:6800/4101821536 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f519c000b90 con 0x7f51a406e990 2024-02-10T07:39:16.802 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.802+0000 7f51ae7fc700 1 -- 172.21.15.70:0/2026428545 <== mgr.34502 v1:172.21.15.70:6800/4101821536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384489 (unknown 2965378022 0 1759718548) 0x7f519c000b90 con 0x7f51a406e990 2024-02-10T07:39:16.808 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:39:16.810 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.805+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 >> v1:172.21.15.70:6800/4101821536 conn(0x7f51a406e990 legacy=0x7f51a4070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.805+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 >> v1:172.21.15.70:6790/0 conn(0x7f51b8109800 legacy=0x7f51b81a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.806+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 shutdown_connections 2024-02-10T07:39:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.806+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 >> 172.21.15.70:0/2026428545 conn(0x7f51b8076e50 msgr2=0x7f51b810bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.806+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 shutdown_connections 2024-02-10T07:39:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:16.806+0000 7f51bf9e1700 1 -- 172.21.15.70:0/2026428545 wait complete. 2024-02-10T07:39:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-02-10T07:39:17.739 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:17 smithi070 ceph-mon[39923]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:17.794 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:17 smithi070 ceph-mon[29548]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:17.795 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:17 smithi155 ceph-mon[32688]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:18.577 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:18 smithi070 ceph-mon[29548]: from='client.34649 v1:172.21.15.70:0/2026428545' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:39:18.577 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:18 smithi070 ceph-mon[39923]: from='client.34649 v1:172.21.15.70:0/2026428545' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:39:18.607 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-02-10T07:39:15.761087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1001,"num_read_kb":847,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":236080,"kb_used_data":7452,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725136,"statfs":{"total":767960285184,"available":767718539264,"internally_reserved":0,"allocated":7630848,"data_stored":4046444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002647"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747873+0000","last_change":"2024-02-10T07:38:04.186647+0000","last_active":"2024-02-10T07:38:37.747873+0000","last_peered":"2024-02-10T07:38:37.747873+0000","last_clean":"2024-02-10T07:38:37.747873+0000","last_became_active":"2024-02-10T07:38:04.186088+0000","last_became_peered":"2024-02-10T07:38:04.186088+0000","last_unstale":"2024-02-10T07:38:37.747873+0000","last_undegraded":"2024-02-10T07:38:37.747873+0000","last_fullsized":"2024-02-10T07:38:37.747873+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:21:31.816616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849094+0000","last_change":"2024-02-10T07:38:02.184576+0000","last_active":"2024-02-10T07:38:37.849094+0000","last_peered":"2024-02-10T07:38:37.849094+0000","last_clean":"2024-02-10T07:38:37.849094+0000","last_became_active":"2024-02-10T07:38:02.184185+0000","last_became_peered":"2024-02-10T07:38:02.184185+0000","last_unstale":"2024-02-10T07:38:37.849094+0000","last_undegraded":"2024-02-10T07:38:37.849094+0000","last_fullsized":"2024-02-10T07:38:37.849094+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:50:50.851110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849140+0000","last_change":"2024-02-10T07:37:58.158178+0000","last_active":"2024-02-10T07:38:37.849140+0000","last_peered":"2024-02-10T07:38:37.849140+0000","last_clean":"2024-02-10T07:38:37.849140+0000","last_became_active":"2024-02-10T07:37:58.157812+0000","last_became_peered":"2024-02-10T07:37:58.157812+0000","last_unstale":"2024-02-10T07:38:37.849140+0000","last_undegraded":"2024-02-10T07:38:37.849140+0000","last_fullsized":"2024-02-10T07:38:37.849140+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:03:36.743454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747462+0000","last_change":"2024-02-10T07:38:00.162568+0000","last_active":"2024-02-10T07:38:37.747462+0000","last_peered":"2024-02-10T07:38:37.747462+0000","last_clean":"2024-02-10T07:38:37.747462+0000","last_became_active":"2024-02-10T07:38:00.162210+0000","last_became_peered":"2024-02-10T07:38:00.162210+0000","last_unstale":"2024-02-10T07:38:37.747462+0000","last_undegraded":"2024-02-10T07:38:37.747462+0000","last_fullsized":"2024-02-10T07:38:37.747462+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:03:39.071261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848975+0000","last_change":"2024-02-10T07:38:04.563853+0000","last_active":"2024-02-10T07:38:37.848975+0000","last_peered":"2024-02-10T07:38:37.848975+0000","last_clean":"2024-02-10T07:38:37.848975+0000","last_became_active":"2024-02-10T07:38:04.563512+0000","last_became_peered":"2024-02-10T07:38:04.563512+0000","last_unstale":"2024-02-10T07:38:37.848975+0000","last_undegraded":"2024-02-10T07:38:37.848975+0000","last_fullsized":"2024-02-10T07:38:37.848975+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:45.876177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747462+0000","last_change":"2024-02-10T07:38:02.185374+0000","last_active":"2024-02-10T07:38:37.747462+0000","last_peered":"2024-02-10T07:38:37.747462+0000","last_clean":"2024-02-10T07:38:37.747462+0000","last_became_active":"2024-02-10T07:38:02.184300+0000","last_became_peered":"2024-02-10T07:38:02.184300+0000","last_unstale":"2024-02-10T07:38:37.747462+0000","last_undegraded":"2024-02-10T07:38:37.747462+0000","last_fullsized":"2024-02-10T07:38:37.747462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:06:37.024983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851305+0000","last_change":"2024-02-10T07:37:58.154555+0000","last_active":"2024-02-10T07:38:37.851305+0000","last_peered":"2024-02-10T07:38:37.851305+0000","last_clean":"2024-02-10T07:38:37.851305+0000","last_became_active":"2024-02-10T07:37:58.154264+0000","last_became_peered":"2024-02-10T07:37:58.154264+0000","last_unstale":"2024-02-10T07:38:37.851305+0000","last_undegraded":"2024-02-10T07:38:37.851305+0000","last_fullsized":"2024-02-10T07:38:37.851305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:05:38.202189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849817+0000","last_change":"2024-02-10T07:38:00.168546+0000","last_active":"2024-02-10T07:38:37.849817+0000","last_peered":"2024-02-10T07:38:37.849817+0000","last_clean":"2024-02-10T07:38:37.849817+0000","last_became_active":"2024-02-10T07:38:00.168115+0000","last_became_peered":"2024-02-10T07:38:00.168115+0000","last_unstale":"2024-02-10T07:38:37.849817+0000","last_undegraded":"2024-02-10T07:38:37.849817+0000","last_fullsized":"2024-02-10T07:38:37.849817+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:37:54.192045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848145+0000","last_change":"2024-02-10T07:38:04.191353+0000","last_active":"2024-02-10T07:38:37.848145+0000","last_peered":"2024-02-10T07:38:37.848145+0000","last_clean":"2024-02-10T07:38:37.848145+0000","last_became_active":"2024-02-10T07:38:04.191014+0000","last_became_peered":"2024-02-10T07:38:04.191014+0000","last_unstale":"2024-02-10T07:38:37.848145+0000","last_undegraded":"2024-02-10T07:38:37.848145+0000","last_fullsized":"2024-02-10T07:38:37.848145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:58:43.576706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834650+0000","last_change":"2024-02-10T07:38:02.185353+0000","last_active":"2024-02-10T07:38:37.834650+0000","last_peered":"2024-02-10T07:38:37.834650+0000","last_clean":"2024-02-10T07:38:37.834650+0000","last_became_active":"2024-02-10T07:38:02.184968+0000","last_became_peered":"2024-02-10T07:38:02.184968+0000","last_unstale":"2024-02-10T07:38:37.834650+0000","last_undegraded":"2024-02-10T07:38:37.834650+0000","last_fullsized":"2024-02-10T07:38:37.834650+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:04:41.595322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834740+0000","last_change":"2024-02-10T07:37:58.153195+0000","last_active":"2024-02-10T07:38:37.834740+0000","last_peered":"2024-02-10T07:38:37.834740+0000","last_clean":"2024-02-10T07:38:37.834740+0000","last_became_active":"2024-02-10T07:37:58.152916+0000","last_became_peered":"2024-02-10T07:37:58.152916+0000","last_unstale":"2024-02-10T07:38:37.834740+0000","last_undegraded":"2024-02-10T07:38:37.834740+0000","last_fullsized":"2024-02-10T07:38:37.834740+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:20:15.789480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748113+0000","last_change":"2024-02-10T07:38:00.164083+0000","last_active":"2024-02-10T07:38:37.748113+0000","last_peered":"2024-02-10T07:38:37.748113+0000","last_clean":"2024-02-10T07:38:37.748113+0000","last_became_active":"2024-02-10T07:38:00.163762+0000","last_became_peered":"2024-02-10T07:38:00.163762+0000","last_unstale":"2024-02-10T07:38:37.748113+0000","last_undegraded":"2024-02-10T07:38:37.748113+0000","last_fullsized":"2024-02-10T07:38:37.748113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:33:52.219839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834294+0000","last_change":"2024-02-10T07:38:04.193340+0000","last_active":"2024-02-10T07:38:37.834294+0000","last_peered":"2024-02-10T07:38:37.834294+0000","last_clean":"2024-02-10T07:38:37.834294+0000","last_became_active":"2024-02-10T07:38:04.193003+0000","last_became_peered":"2024-02-10T07:38:04.193003+0000","last_unstale":"2024-02-10T07:38:37.834294+0000","last_undegraded":"2024-02-10T07:38:37.834294+0000","last_fullsized":"2024-02-10T07:38:37.834294+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:27:01.077971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844052+0000","last_change":"2024-02-10T07:38:02.186017+0000","last_active":"2024-02-10T07:38:37.844052+0000","last_peered":"2024-02-10T07:38:37.844052+0000","last_clean":"2024-02-10T07:38:37.844052+0000","last_became_active":"2024-02-10T07:38:02.185713+0000","last_became_peered":"2024-02-10T07:38:02.185713+0000","last_unstale":"2024-02-10T07:38:37.844052+0000","last_undegraded":"2024-02-10T07:38:37.844052+0000","last_fullsized":"2024-02-10T07:38:37.844052+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:19:31.424409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834664+0000","last_change":"2024-02-10T07:37:58.150762+0000","last_active":"2024-02-10T07:38:37.834664+0000","last_peered":"2024-02-10T07:38:37.834664+0000","last_clean":"2024-02-10T07:38:37.834664+0000","last_became_active":"2024-02-10T07:37:58.150477+0000","last_became_peered":"2024-02-10T07:37:58.150477+0000","last_unstale":"2024-02-10T07:38:37.834664+0000","last_undegraded":"2024-02-10T07:38:37.834664+0000","last_fullsized":"2024-02-10T07:38:37.834664+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:44:47.243565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"69'5","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748915+0000","last_change":"2024-02-10T07:38:00.169434+0000","last_active":"2024-02-10T07:38:37.748915+0000","last_peered":"2024-02-10T07:38:37.748915+0000","last_clean":"2024-02-10T07:38:37.748915+0000","last_became_active":"2024-02-10T07:38:00.169148+0000","last_became_peered":"2024-02-10T07:38:00.169148+0000","last_unstale":"2024-02-10T07:38:37.748915+0000","last_undegraded":"2024-02-10T07:38:37.748915+0000","last_fullsized":"2024-02-10T07:38:37.748915+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:57:18.133325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.750255+0000","last_change":"2024-02-10T07:38:04.561736+0000","last_active":"2024-02-10T07:38:37.750255+0000","last_peered":"2024-02-10T07:38:37.750255+0000","last_clean":"2024-02-10T07:38:37.750255+0000","last_became_active":"2024-02-10T07:38:04.561400+0000","last_became_peered":"2024-02-10T07:38:04.561400+0000","last_unstale":"2024-02-10T07:38:37.750255+0000","last_undegraded":"2024-02-10T07:38:37.750255+0000","last_fullsized":"2024-02-10T07:38:37.750255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:32:05.994039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848395+0000","last_change":"2024-02-10T07:38:02.183003+0000","last_active":"2024-02-10T07:38:37.848395+0000","last_peered":"2024-02-10T07:38:37.848395+0000","last_clean":"2024-02-10T07:38:37.848395+0000","last_became_active":"2024-02-10T07:38:02.182700+0000","last_became_peered":"2024-02-10T07:38:02.182700+0000","last_unstale":"2024-02-10T07:38:37.848395+0000","last_undegraded":"2024-02-10T07:38:37.848395+0000","last_fullsized":"2024-02-10T07:38:37.848395+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:23:19.764457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850257+0000","last_change":"2024-02-10T07:37:58.158806+0000","last_active":"2024-02-10T07:38:37.850257+0000","last_peered":"2024-02-10T07:38:37.850257+0000","last_clean":"2024-02-10T07:38:37.850257+0000","last_became_active":"2024-02-10T07:37:58.158534+0000","last_became_peered":"2024-02-10T07:37:58.158534+0000","last_unstale":"2024-02-10T07:38:37.850257+0000","last_undegraded":"2024-02-10T07:38:37.850257+0000","last_fullsized":"2024-02-10T07:38:37.850257+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:31:26.321110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745044+0000","last_change":"2024-02-10T07:38:00.170213+0000","last_active":"2024-02-10T07:38:37.745044+0000","last_peered":"2024-02-10T07:38:37.745044+0000","last_clean":"2024-02-10T07:38:37.745044+0000","last_became_active":"2024-02-10T07:38:00.169807+0000","last_became_peered":"2024-02-10T07:38:00.169807+0000","last_unstale":"2024-02-10T07:38:37.745044+0000","last_undegraded":"2024-02-10T07:38:37.745044+0000","last_fullsized":"2024-02-10T07:38:37.745044+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:15:47.790155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851845+0000","last_change":"2024-02-10T07:38:04.563035+0000","last_active":"2024-02-10T07:38:37.851845+0000","last_peered":"2024-02-10T07:38:37.851845+0000","last_clean":"2024-02-10T07:38:37.851845+0000","last_became_active":"2024-02-10T07:38:04.562573+0000","last_became_peered":"2024-02-10T07:38:04.562573+0000","last_unstale":"2024-02-10T07:38:37.851845+0000","last_undegraded":"2024-02-10T07:38:37.851845+0000","last_fullsized":"2024-02-10T07:38:37.851845+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:13:10.575665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748177+0000","last_change":"2024-02-10T07:38:02.186897+0000","last_active":"2024-02-10T07:38:37.748177+0000","last_peered":"2024-02-10T07:38:37.748177+0000","last_clean":"2024-02-10T07:38:37.748177+0000","last_became_active":"2024-02-10T07:38:02.186606+0000","last_became_peered":"2024-02-10T07:38:02.186606+0000","last_unstale":"2024-02-10T07:38:37.748177+0000","last_undegraded":"2024-02-10T07:38:37.748177+0000","last_fullsized":"2024-02-10T07:38:37.748177+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:30:58.044937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748225+0000","last_change":"2024-02-10T07:37:58.157435+0000","last_active":"2024-02-10T07:38:37.748225+0000","last_peered":"2024-02-10T07:38:37.748225+0000","last_clean":"2024-02-10T07:38:37.748225+0000","last_became_active":"2024-02-10T07:37:58.157142+0000","last_became_peered":"2024-02-10T07:37:58.157142+0000","last_unstale":"2024-02-10T07:38:37.748225+0000","last_undegraded":"2024-02-10T07:38:37.748225+0000","last_fullsized":"2024-02-10T07:38:37.748225+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:12.624671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"69'12","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850472+0000","last_change":"2024-02-10T07:38:00.169085+0000","last_active":"2024-02-10T07:38:37.850472+0000","last_peered":"2024-02-10T07:38:37.850472+0000","last_clean":"2024-02-10T07:38:37.850472+0000","last_became_active":"2024-02-10T07:38:00.168028+0000","last_became_peered":"2024-02-10T07:38:00.168028+0000","last_unstale":"2024-02-10T07:38:37.850472+0000","last_undegraded":"2024-02-10T07:38:37.850472+0000","last_fullsized":"2024-02-10T07:38:37.850472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:18:00.568320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844353+0000","last_change":"2024-02-10T07:38:04.189744+0000","last_active":"2024-02-10T07:38:37.844353+0000","last_peered":"2024-02-10T07:38:37.844353+0000","last_clean":"2024-02-10T07:38:37.844353+0000","last_became_active":"2024-02-10T07:38:04.189496+0000","last_became_peered":"2024-02-10T07:38:04.189496+0000","last_unstale":"2024-02-10T07:38:37.844353+0000","last_undegraded":"2024-02-10T07:38:37.844353+0000","last_fullsized":"2024-02-10T07:38:37.844353+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:12.837478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211876+0000","last_change":"2024-02-10T07:38:02.187005+0000","last_active":"2024-02-10T07:39:08.211876+0000","last_peered":"2024-02-10T07:39:08.211876+0000","last_clean":"2024-02-10T07:39:08.211876+0000","last_became_active":"2024-02-10T07:38:02.186671+0000","last_became_peered":"2024-02-10T07:38:02.186671+0000","last_unstale":"2024-02-10T07:39:08.211876+0000","last_undegraded":"2024-02-10T07:39:08.211876+0000","last_fullsized":"2024-02-10T07:39:08.211876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:21.033127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849175+0000","last_change":"2024-02-10T07:37:58.155111+0000","last_active":"2024-02-10T07:38:37.849175+0000","last_peered":"2024-02-10T07:38:37.849175+0000","last_clean":"2024-02-10T07:38:37.849175+0000","last_became_active":"2024-02-10T07:37:58.154702+0000","last_became_peered":"2024-02-10T07:37:58.154702+0000","last_unstale":"2024-02-10T07:38:37.849175+0000","last_undegraded":"2024-02-10T07:38:37.849175+0000","last_fullsized":"2024-02-10T07:38:37.849175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:50:38.943235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849587+0000","last_change":"2024-02-10T07:38:00.767811+0000","last_active":"2024-02-10T07:38:37.849587+0000","last_peered":"2024-02-10T07:38:37.849587+0000","last_clean":"2024-02-10T07:38:37.849587+0000","last_became_active":"2024-02-10T07:38:00.767522+0000","last_became_peered":"2024-02-10T07:38:00.767522+0000","last_unstale":"2024-02-10T07:38:37.849587+0000","last_undegraded":"2024-02-10T07:38:37.849587+0000","last_fullsized":"2024-02-10T07:38:37.849587+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:31:21.312717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848518+0000","last_change":"2024-02-10T07:38:04.187165+0000","last_active":"2024-02-10T07:38:37.848518+0000","last_peered":"2024-02-10T07:38:37.848518+0000","last_clean":"2024-02-10T07:38:37.848518+0000","last_became_active":"2024-02-10T07:38:04.186402+0000","last_became_peered":"2024-02-10T07:38:04.186402+0000","last_unstale":"2024-02-10T07:38:37.848518+0000","last_undegraded":"2024-02-10T07:38:37.848518+0000","last_fullsized":"2024-02-10T07:38:37.848518+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:07:33.500379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849905+0000","last_change":"2024-02-10T07:38:02.188677+0000","last_active":"2024-02-10T07:38:37.849905+0000","last_peered":"2024-02-10T07:38:37.849905+0000","last_clean":"2024-02-10T07:38:37.849905+0000","last_became_active":"2024-02-10T07:38:02.188391+0000","last_became_peered":"2024-02-10T07:38:02.188391+0000","last_unstale":"2024-02-10T07:38:37.849905+0000","last_undegraded":"2024-02-10T07:38:37.849905+0000","last_fullsized":"2024-02-10T07:38:37.849905+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:49:27.901890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847848+0000","last_change":"2024-02-10T07:37:58.151815+0000","last_active":"2024-02-10T07:38:37.847848+0000","last_peered":"2024-02-10T07:38:37.847848+0000","last_clean":"2024-02-10T07:38:37.847848+0000","last_became_active":"2024-02-10T07:37:58.151572+0000","last_became_peered":"2024-02-10T07:37:58.151572+0000","last_unstale":"2024-02-10T07:38:37.847848+0000","last_undegraded":"2024-02-10T07:38:37.847848+0000","last_fullsized":"2024-02-10T07:38:37.847848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:57:09.209275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"69'11","reported_seq":66,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847970+0000","last_change":"2024-02-10T07:38:28.787238+0000","last_active":"2024-02-10T07:38:37.847970+0000","last_peered":"2024-02-10T07:38:37.847970+0000","last_clean":"2024-02-10T07:38:37.847970+0000","last_became_active":"2024-02-10T07:38:28.776345+0000","last_became_peered":"2024-02-10T07:38:28.776345+0000","last_unstale":"2024-02-10T07:38:37.847970+0000","last_undegraded":"2024-02-10T07:38:37.847970+0000","last_fullsized":"2024-02-10T07:38:37.847970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:55:48.034095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.843999+0000","last_change":"2024-02-10T07:38:04.189394+0000","last_active":"2024-02-10T07:38:37.843999+0000","last_peered":"2024-02-10T07:38:37.843999+0000","last_clean":"2024-02-10T07:38:37.843999+0000","last_became_active":"2024-02-10T07:38:04.189126+0000","last_became_peered":"2024-02-10T07:38:04.189126+0000","last_unstale":"2024-02-10T07:38:37.843999+0000","last_undegraded":"2024-02-10T07:38:37.843999+0000","last_fullsized":"2024-02-10T07:38:37.843999+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:06:49.453471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847377+0000","last_change":"2024-02-10T07:38:02.183392+0000","last_active":"2024-02-10T07:38:37.847377+0000","last_peered":"2024-02-10T07:38:37.847377+0000","last_clean":"2024-02-10T07:38:37.847377+0000","last_became_active":"2024-02-10T07:38:02.183011+0000","last_became_peered":"2024-02-10T07:38:02.183011+0000","last_unstale":"2024-02-10T07:38:37.847377+0000","last_undegraded":"2024-02-10T07:38:37.847377+0000","last_fullsized":"2024-02-10T07:38:37.847377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:37:04.557621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849039+0000","last_change":"2024-02-10T07:37:58.153169+0000","last_active":"2024-02-10T07:38:37.849039+0000","last_peered":"2024-02-10T07:38:37.849039+0000","last_clean":"2024-02-10T07:38:37.849039+0000","last_became_active":"2024-02-10T07:37:58.152890+0000","last_became_peered":"2024-02-10T07:37:58.152890+0000","last_unstale":"2024-02-10T07:38:37.849039+0000","last_undegraded":"2024-02-10T07:38:37.849039+0000","last_fullsized":"2024-02-10T07:38:37.849039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:50:21.140869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847953+0000","last_change":"2024-02-10T07:38:00.172054+0000","last_active":"2024-02-10T07:38:37.847953+0000","last_peered":"2024-02-10T07:38:37.847953+0000","last_clean":"2024-02-10T07:38:37.847953+0000","last_became_active":"2024-02-10T07:38:00.171747+0000","last_became_peered":"2024-02-10T07:38:00.171747+0000","last_unstale":"2024-02-10T07:38:37.847953+0000","last_undegraded":"2024-02-10T07:38:37.847953+0000","last_fullsized":"2024-02-10T07:38:37.847953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:29:06.097734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848792+0000","last_change":"2024-02-10T07:38:04.562960+0000","last_active":"2024-02-10T07:38:37.848792+0000","last_peered":"2024-02-10T07:38:37.848792+0000","last_clean":"2024-02-10T07:38:37.848792+0000","last_became_active":"2024-02-10T07:38:04.562557+0000","last_became_peered":"2024-02-10T07:38:04.562557+0000","last_unstale":"2024-02-10T07:38:37.848792+0000","last_undegraded":"2024-02-10T07:38:37.848792+0000","last_fullsized":"2024-02-10T07:38:37.848792+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:46:44.975257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844207+0000","last_change":"2024-02-10T07:38:02.190989+0000","last_active":"2024-02-10T07:38:37.844207+0000","last_peered":"2024-02-10T07:38:37.844207+0000","last_clean":"2024-02-10T07:38:37.844207+0000","last_became_active":"2024-02-10T07:38:02.190505+0000","last_became_peered":"2024-02-10T07:38:02.190505+0000","last_unstale":"2024-02-10T07:38:37.844207+0000","last_undegraded":"2024-02-10T07:38:37.844207+0000","last_fullsized":"2024-02-10T07:38:37.844207+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:09:43.496965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834954+0000","last_change":"2024-02-10T07:37:58.150118+0000","last_active":"2024-02-10T07:38:37.834954+0000","last_peered":"2024-02-10T07:38:37.834954+0000","last_clean":"2024-02-10T07:38:37.834954+0000","last_became_active":"2024-02-10T07:37:58.149812+0000","last_became_peered":"2024-02-10T07:37:58.149812+0000","last_unstale":"2024-02-10T07:38:37.834954+0000","last_undegraded":"2024-02-10T07:38:37.834954+0000","last_fullsized":"2024-02-10T07:38:37.834954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:24:20.526839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849672+0000","last_change":"2024-02-10T07:38:00.765844+0000","last_active":"2024-02-10T07:38:37.849672+0000","last_peered":"2024-02-10T07:38:37.849672+0000","last_clean":"2024-02-10T07:38:37.849672+0000","last_became_active":"2024-02-10T07:38:00.765467+0000","last_became_peered":"2024-02-10T07:38:00.765467+0000","last_unstale":"2024-02-10T07:38:37.849672+0000","last_undegraded":"2024-02-10T07:38:37.849672+0000","last_fullsized":"2024-02-10T07:38:37.849672+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:14:19.211109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847715+0000","last_change":"2024-02-10T07:38:04.190076+0000","last_active":"2024-02-10T07:38:37.847715+0000","last_peered":"2024-02-10T07:38:37.847715+0000","last_clean":"2024-02-10T07:38:37.847715+0000","last_became_active":"2024-02-10T07:38:04.189670+0000","last_became_peered":"2024-02-10T07:38:04.189670+0000","last_unstale":"2024-02-10T07:38:37.847715+0000","last_undegraded":"2024-02-10T07:38:37.847715+0000","last_fullsized":"2024-02-10T07:38:37.847715+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:42:20.254668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849059+0000","last_change":"2024-02-10T07:38:02.188571+0000","last_active":"2024-02-10T07:38:37.849059+0000","last_peered":"2024-02-10T07:38:37.849059+0000","last_clean":"2024-02-10T07:38:37.849059+0000","last_became_active":"2024-02-10T07:38:02.188246+0000","last_became_peered":"2024-02-10T07:38:02.188246+0000","last_unstale":"2024-02-10T07:38:37.849059+0000","last_undegraded":"2024-02-10T07:38:37.849059+0000","last_fullsized":"2024-02-10T07:38:37.849059+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:32:33.117903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844509+0000","last_change":"2024-02-10T07:37:58.153978+0000","last_active":"2024-02-10T07:38:37.844509+0000","last_peered":"2024-02-10T07:38:37.844509+0000","last_clean":"2024-02-10T07:38:37.844509+0000","last_became_active":"2024-02-10T07:37:58.153407+0000","last_became_peered":"2024-02-10T07:37:58.153407+0000","last_unstale":"2024-02-10T07:38:37.844509+0000","last_undegraded":"2024-02-10T07:38:37.844509+0000","last_fullsized":"2024-02-10T07:38:37.844509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:40:28.719385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848397+0000","last_change":"2024-02-10T07:38:00.164466+0000","last_active":"2024-02-10T07:38:37.848397+0000","last_peered":"2024-02-10T07:38:37.848397+0000","last_clean":"2024-02-10T07:38:37.848397+0000","last_became_active":"2024-02-10T07:38:00.162718+0000","last_became_peered":"2024-02-10T07:38:00.162718+0000","last_unstale":"2024-02-10T07:38:37.848397+0000","last_undegraded":"2024-02-10T07:38:37.848397+0000","last_fullsized":"2024-02-10T07:38:37.848397+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:15:16.350207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849363+0000","last_change":"2024-02-10T07:38:04.197091+0000","last_active":"2024-02-10T07:38:37.849363+0000","last_peered":"2024-02-10T07:38:37.849363+0000","last_clean":"2024-02-10T07:38:37.849363+0000","last_became_active":"2024-02-10T07:38:04.196773+0000","last_became_peered":"2024-02-10T07:38:04.196773+0000","last_unstale":"2024-02-10T07:38:37.849363+0000","last_undegraded":"2024-02-10T07:38:37.849363+0000","last_fullsized":"2024-02-10T07:38:37.849363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:46:54.401235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848312+0000","last_change":"2024-02-10T07:38:02.185433+0000","last_active":"2024-02-10T07:38:37.848312+0000","last_peered":"2024-02-10T07:38:37.848312+0000","last_clean":"2024-02-10T07:38:37.848312+0000","last_became_active":"2024-02-10T07:38:02.185111+0000","last_became_peered":"2024-02-10T07:38:02.185111+0000","last_unstale":"2024-02-10T07:38:37.848312+0000","last_undegraded":"2024-02-10T07:38:37.848312+0000","last_fullsized":"2024-02-10T07:38:37.848312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:09.343575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748416+0000","last_change":"2024-02-10T07:37:58.152079+0000","last_active":"2024-02-10T07:38:37.748416+0000","last_peered":"2024-02-10T07:38:37.748416+0000","last_clean":"2024-02-10T07:38:37.748416+0000","last_became_active":"2024-02-10T07:37:58.151811+0000","last_became_peered":"2024-02-10T07:37:58.151811+0000","last_unstale":"2024-02-10T07:38:37.748416+0000","last_undegraded":"2024-02-10T07:38:37.748416+0000","last_fullsized":"2024-02-10T07:38:37.748416+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:48:42.546171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748745+0000","last_change":"2024-02-10T07:38:00.168712+0000","last_active":"2024-02-10T07:38:37.748745+0000","last_peered":"2024-02-10T07:38:37.748745+0000","last_clean":"2024-02-10T07:38:37.748745+0000","last_became_active":"2024-02-10T07:38:00.167745+0000","last_became_peered":"2024-02-10T07:38:00.167745+0000","last_unstale":"2024-02-10T07:38:37.748745+0000","last_undegraded":"2024-02-10T07:38:37.748745+0000","last_fullsized":"2024-02-10T07:38:37.748745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:47:11.415857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849273+0000","last_change":"2024-02-10T07:38:04.197103+0000","last_active":"2024-02-10T07:38:37.849273+0000","last_peered":"2024-02-10T07:38:37.849273+0000","last_clean":"2024-02-10T07:38:37.849273+0000","last_became_active":"2024-02-10T07:38:04.196769+0000","last_became_peered":"2024-02-10T07:38:04.196769+0000","last_unstale":"2024-02-10T07:38:37.849273+0000","last_undegraded":"2024-02-10T07:38:37.849273+0000","last_fullsized":"2024-02-10T07:38:37.849273+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:54:44.258613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211896+0000","last_change":"2024-02-10T07:38:02.185745+0000","last_active":"2024-02-10T07:39:08.211896+0000","last_peered":"2024-02-10T07:39:08.211896+0000","last_clean":"2024-02-10T07:39:08.211896+0000","last_became_active":"2024-02-10T07:38:02.185473+0000","last_became_peered":"2024-02-10T07:38:02.185473+0000","last_unstale":"2024-02-10T07:39:08.211896+0000","last_undegraded":"2024-02-10T07:39:08.211896+0000","last_fullsized":"2024-02-10T07:39:08.211896+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:36:59.933516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844133+0000","last_change":"2024-02-10T07:37:58.155405+0000","last_active":"2024-02-10T07:38:37.844133+0000","last_peered":"2024-02-10T07:38:37.844133+0000","last_clean":"2024-02-10T07:38:37.844133+0000","last_became_active":"2024-02-10T07:37:58.155112+0000","last_became_peered":"2024-02-10T07:37:58.155112+0000","last_unstale":"2024-02-10T07:38:37.844133+0000","last_undegraded":"2024-02-10T07:38:37.844133+0000","last_fullsized":"2024-02-10T07:38:37.844133+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:34:44.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850742+0000","last_change":"2024-02-10T07:38:00.765860+0000","last_active":"2024-02-10T07:38:37.850742+0000","last_peered":"2024-02-10T07:38:37.850742+0000","last_clean":"2024-02-10T07:38:37.850742+0000","last_became_active":"2024-02-10T07:38:00.765550+0000","last_became_peered":"2024-02-10T07:38:00.765550+0000","last_unstale":"2024-02-10T07:38:37.850742+0000","last_undegraded":"2024-02-10T07:38:37.850742+0000","last_fullsized":"2024-02-10T07:38:37.850742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:18:57.658269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.749941+0000","last_change":"2024-02-10T07:38:04.189975+0000","last_active":"2024-02-10T07:38:37.749941+0000","last_peered":"2024-02-10T07:38:37.749941+0000","last_clean":"2024-02-10T07:38:37.749941+0000","last_became_active":"2024-02-10T07:38:04.189674+0000","last_became_peered":"2024-02-10T07:38:04.189674+0000","last_unstale":"2024-02-10T07:38:37.749941+0000","last_undegraded":"2024-02-10T07:38:37.749941+0000","last_fullsized":"2024-02-10T07:38:37.749941+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:59:42.747712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211596+0000","last_change":"2024-02-10T07:38:02.187629+0000","last_active":"2024-02-10T07:39:08.211596+0000","last_peered":"2024-02-10T07:39:08.211596+0000","last_clean":"2024-02-10T07:39:08.211596+0000","last_became_active":"2024-02-10T07:38:02.187277+0000","last_became_peered":"2024-02-10T07:38:02.187277+0000","last_unstale":"2024-02-10T07:39:08.211596+0000","last_undegraded":"2024-02-10T07:39:08.211596+0000","last_fullsized":"2024-02-10T07:39:08.211596+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:54:07.636346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848402+0000","last_change":"2024-02-10T07:37:58.159222+0000","last_active":"2024-02-10T07:38:37.848402+0000","last_peered":"2024-02-10T07:38:37.848402+0000","last_clean":"2024-02-10T07:38:37.848402+0000","last_became_active":"2024-02-10T07:37:58.158523+0000","last_became_peered":"2024-02-10T07:37:58.158523+0000","last_unstale":"2024-02-10T07:38:37.848402+0000","last_undegraded":"2024-02-10T07:38:37.848402+0000","last_fullsized":"2024-02-10T07:38:37.848402+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:08:28.933514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.835050+0000","last_change":"2024-02-10T07:38:00.768575+0000","last_active":"2024-02-10T07:38:37.835050+0000","last_peered":"2024-02-10T07:38:37.835050+0000","last_clean":"2024-02-10T07:38:37.835050+0000","last_became_active":"2024-02-10T07:38:00.768184+0000","last_became_peered":"2024-02-10T07:38:00.768184+0000","last_unstale":"2024-02-10T07:38:37.835050+0000","last_undegraded":"2024-02-10T07:38:37.835050+0000","last_fullsized":"2024-02-10T07:38:37.835050+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:18:34.337454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.746049+0000","last_change":"2024-02-10T07:38:04.192255+0000","last_active":"2024-02-10T07:38:37.746049+0000","last_peered":"2024-02-10T07:38:37.746049+0000","last_clean":"2024-02-10T07:38:37.746049+0000","last_became_active":"2024-02-10T07:38:04.191959+0000","last_became_peered":"2024-02-10T07:38:04.191959+0000","last_unstale":"2024-02-10T07:38:37.746049+0000","last_undegraded":"2024-02-10T07:38:37.746049+0000","last_fullsized":"2024-02-10T07:38:37.746049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:47:40.619894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851740+0000","last_change":"2024-02-10T07:38:02.188897+0000","last_active":"2024-02-10T07:38:37.851740+0000","last_peered":"2024-02-10T07:38:37.851740+0000","last_clean":"2024-02-10T07:38:37.851740+0000","last_became_active":"2024-02-10T07:38:02.188576+0000","last_became_peered":"2024-02-10T07:38:02.188576+0000","last_unstale":"2024-02-10T07:38:37.851740+0000","last_undegraded":"2024-02-10T07:38:37.851740+0000","last_fullsized":"2024-02-10T07:38:37.851740+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:11:27.500027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844069+0000","last_change":"2024-02-10T07:37:58.155052+0000","last_active":"2024-02-10T07:38:37.844069+0000","last_peered":"2024-02-10T07:38:37.844069+0000","last_clean":"2024-02-10T07:38:37.844069+0000","last_became_active":"2024-02-10T07:37:58.154767+0000","last_became_peered":"2024-02-10T07:37:58.154767+0000","last_unstale":"2024-02-10T07:38:37.844069+0000","last_undegraded":"2024-02-10T07:38:37.844069+0000","last_fullsized":"2024-02-10T07:38:37.844069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:32:26.672068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849712+0000","last_change":"2024-02-10T07:38:00.767800+0000","last_active":"2024-02-10T07:38:37.849712+0000","last_peered":"2024-02-10T07:38:37.849712+0000","last_clean":"2024-02-10T07:38:37.849712+0000","last_became_active":"2024-02-10T07:38:00.767513+0000","last_became_peered":"2024-02-10T07:38:00.767513+0000","last_unstale":"2024-02-10T07:38:37.849712+0000","last_undegraded":"2024-02-10T07:38:37.849712+0000","last_fullsized":"2024-02-10T07:38:37.849712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:13:09.563969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844455+0000","last_change":"2024-02-10T07:38:04.564998+0000","last_active":"2024-02-10T07:38:37.844455+0000","last_peered":"2024-02-10T07:38:37.844455+0000","last_clean":"2024-02-10T07:38:37.844455+0000","last_became_active":"2024-02-10T07:38:04.564088+0000","last_became_peered":"2024-02-10T07:38:04.564088+0000","last_unstale":"2024-02-10T07:38:37.844455+0000","last_undegraded":"2024-02-10T07:38:37.844455+0000","last_fullsized":"2024-02-10T07:38:37.844455+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:52:22.234034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834953+0000","last_change":"2024-02-10T07:38:02.184168+0000","last_active":"2024-02-10T07:38:37.834953+0000","last_peered":"2024-02-10T07:38:37.834953+0000","last_clean":"2024-02-10T07:38:37.834953+0000","last_became_active":"2024-02-10T07:38:02.182892+0000","last_became_peered":"2024-02-10T07:38:02.182892+0000","last_unstale":"2024-02-10T07:38:37.834953+0000","last_undegraded":"2024-02-10T07:38:37.834953+0000","last_fullsized":"2024-02-10T07:38:37.834953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:06:44.551793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847320+0000","last_change":"2024-02-10T07:37:58.150290+0000","last_active":"2024-02-10T07:38:37.847320+0000","last_peered":"2024-02-10T07:38:37.847320+0000","last_clean":"2024-02-10T07:38:37.847320+0000","last_became_active":"2024-02-10T07:37:58.149988+0000","last_became_peered":"2024-02-10T07:37:58.149988+0000","last_unstale":"2024-02-10T07:38:37.847320+0000","last_undegraded":"2024-02-10T07:38:37.847320+0000","last_fullsized":"2024-02-10T07:38:37.847320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:08:59.591165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"69'6","reported_seq":38,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851851+0000","last_change":"2024-02-10T07:38:00.765838+0000","last_active":"2024-02-10T07:38:37.851851+0000","last_peered":"2024-02-10T07:38:37.851851+0000","last_clean":"2024-02-10T07:38:37.851851+0000","last_became_active":"2024-02-10T07:38:00.765455+0000","last_became_peered":"2024-02-10T07:38:00.765455+0000","last_unstale":"2024-02-10T07:38:37.851851+0000","last_undegraded":"2024-02-10T07:38:37.851851+0000","last_fullsized":"2024-02-10T07:38:37.851851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:50.099879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834686+0000","last_change":"2024-02-10T07:38:04.564345+0000","last_active":"2024-02-10T07:38:37.834686+0000","last_peered":"2024-02-10T07:38:37.834686+0000","last_clean":"2024-02-10T07:38:37.834686+0000","last_became_active":"2024-02-10T07:38:04.563522+0000","last_became_peered":"2024-02-10T07:38:04.563522+0000","last_unstale":"2024-02-10T07:38:37.834686+0000","last_undegraded":"2024-02-10T07:38:37.834686+0000","last_fullsized":"2024-02-10T07:38:37.834686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:41:36.968641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"62'2","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844658+0000","last_change":"2024-02-10T07:37:58.154342+0000","last_active":"2024-02-10T07:38:37.844658+0000","last_peered":"2024-02-10T07:38:37.844658+0000","last_clean":"2024-02-10T07:38:37.844658+0000","last_became_active":"2024-02-10T07:37:58.154083+0000","last_became_peered":"2024-02-10T07:37:58.154083+0000","last_unstale":"2024-02-10T07:38:37.844658+0000","last_undegraded":"2024-02-10T07:38:37.844658+0000","last_fullsized":"2024-02-10T07:38:37.844658+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:18:34.317864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834388+0000","last_change":"2024-02-10T07:38:00.767983+0000","last_active":"2024-02-10T07:38:37.834388+0000","last_peered":"2024-02-10T07:38:37.834388+0000","last_clean":"2024-02-10T07:38:37.834388+0000","last_became_active":"2024-02-10T07:38:00.767609+0000","last_became_peered":"2024-02-10T07:38:00.767609+0000","last_unstale":"2024-02-10T07:38:37.834388+0000","last_undegraded":"2024-02-10T07:38:37.834388+0000","last_fullsized":"2024-02-10T07:38:37.834388+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:36:44.429584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211886+0000","last_change":"2024-02-10T07:38:02.185099+0000","last_active":"2024-02-10T07:39:08.211886+0000","last_peered":"2024-02-10T07:39:08.211886+0000","last_clean":"2024-02-10T07:39:08.211886+0000","last_became_active":"2024-02-10T07:38:02.184859+0000","last_became_peered":"2024-02-10T07:38:02.184859+0000","last_unstale":"2024-02-10T07:39:08.211886+0000","last_undegraded":"2024-02-10T07:39:08.211886+0000","last_fullsized":"2024-02-10T07:39:08.211886+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:24:27.874093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848886+0000","last_change":"2024-02-10T07:38:04.194119+0000","last_active":"2024-02-10T07:38:37.848886+0000","last_peered":"2024-02-10T07:38:37.848886+0000","last_clean":"2024-02-10T07:38:37.848886+0000","last_became_active":"2024-02-10T07:38:04.193794+0000","last_became_peered":"2024-02-10T07:38:04.193794+0000","last_unstale":"2024-02-10T07:38:37.848886+0000","last_undegraded":"2024-02-10T07:38:37.848886+0000","last_fullsized":"2024-02-10T07:38:37.848886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:59:38.728525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844561+0000","last_change":"2024-02-10T07:37:58.154699+0000","last_active":"2024-02-10T07:38:37.844561+0000","last_peered":"2024-02-10T07:38:37.844561+0000","last_clean":"2024-02-10T07:38:37.844561+0000","last_became_active":"2024-02-10T07:37:58.154367+0000","last_became_peered":"2024-02-10T07:37:58.154367+0000","last_unstale":"2024-02-10T07:38:37.844561+0000","last_undegraded":"2024-02-10T07:38:37.844561+0000","last_fullsized":"2024-02-10T07:38:37.844561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:17:52.744417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"69'12","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748001+0000","last_change":"2024-02-10T07:38:00.169422+0000","last_active":"2024-02-10T07:38:37.748001+0000","last_peered":"2024-02-10T07:38:37.748001+0000","last_clean":"2024-02-10T07:38:37.748001+0000","last_became_active":"2024-02-10T07:38:00.169111+0000","last_became_peered":"2024-02-10T07:38:00.169111+0000","last_unstale":"2024-02-10T07:38:37.748001+0000","last_undegraded":"2024-02-10T07:38:37.748001+0000","last_fullsized":"2024-02-10T07:38:37.748001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:12:36.038623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.744800+0000","last_change":"2024-02-10T07:38:02.184797+0000","last_active":"2024-02-10T07:38:37.744800+0000","last_peered":"2024-02-10T07:38:37.744800+0000","last_clean":"2024-02-10T07:38:37.744800+0000","last_became_active":"2024-02-10T07:38:02.184490+0000","last_became_peered":"2024-02-10T07:38:02.184490+0000","last_unstale":"2024-02-10T07:38:37.744800+0000","last_undegraded":"2024-02-10T07:38:37.744800+0000","last_fullsized":"2024-02-10T07:38:37.744800+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:38:09.240803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.843866+0000","last_change":"2024-02-10T07:38:04.197954+0000","last_active":"2024-02-10T07:38:37.843866+0000","last_peered":"2024-02-10T07:38:37.843866+0000","last_clean":"2024-02-10T07:38:37.843866+0000","last_became_active":"2024-02-10T07:38:04.197694+0000","last_became_peered":"2024-02-10T07:38:04.197694+0000","last_unstale":"2024-02-10T07:38:37.843866+0000","last_undegraded":"2024-02-10T07:38:37.843866+0000","last_fullsized":"2024-02-10T07:38:37.843866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:21:52.258618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.744946+0000","last_change":"2024-02-10T07:38:02.186168+0000","last_active":"2024-02-10T07:38:37.744946+0000","last_peered":"2024-02-10T07:38:37.744946+0000","last_clean":"2024-02-10T07:38:37.744946+0000","last_became_active":"2024-02-10T07:38:02.185819+0000","last_became_peered":"2024-02-10T07:38:02.185819+0000","last_unstale":"2024-02-10T07:38:37.744946+0000","last_undegraded":"2024-02-10T07:38:37.744946+0000","last_fullsized":"2024-02-10T07:38:37.744946+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:08:16.039062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844784+0000","last_change":"2024-02-10T07:37:58.153698+0000","last_active":"2024-02-10T07:38:37.844784+0000","last_peered":"2024-02-10T07:38:37.844784+0000","last_clean":"2024-02-10T07:38:37.844784+0000","last_became_active":"2024-02-10T07:37:58.152990+0000","last_became_peered":"2024-02-10T07:37:58.152990+0000","last_unstale":"2024-02-10T07:38:37.844784+0000","last_undegraded":"2024-02-10T07:38:37.844784+0000","last_fullsized":"2024-02-10T07:38:37.844784+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:58:23.722396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"69'19","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847824+0000","last_change":"2024-02-10T07:38:00.766017+0000","last_active":"2024-02-10T07:38:37.847824+0000","last_peered":"2024-02-10T07:38:37.847824+0000","last_clean":"2024-02-10T07:38:37.847824+0000","last_became_active":"2024-02-10T07:38:00.765596+0000","last_became_peered":"2024-02-10T07:38:00.765596+0000","last_unstale":"2024-02-10T07:38:37.847824+0000","last_undegraded":"2024-02-10T07:38:37.847824+0000","last_fullsized":"2024-02-10T07:38:37.847824+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:21:07.177967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847964+0000","last_change":"2024-02-10T07:38:04.199641+0000","last_active":"2024-02-10T07:38:37.847964+0000","last_peered":"2024-02-10T07:38:37.847964+0000","last_clean":"2024-02-10T07:38:37.847964+0000","last_became_active":"2024-02-10T07:38:04.199274+0000","last_became_peered":"2024-02-10T07:38:04.199274+0000","last_unstale":"2024-02-10T07:38:37.847964+0000","last_undegraded":"2024-02-10T07:38:37.847964+0000","last_fullsized":"2024-02-10T07:38:37.847964+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:34:12.633850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745319+0000","last_change":"2024-02-10T07:38:02.185126+0000","last_active":"2024-02-10T07:38:37.745319+0000","last_peered":"2024-02-10T07:38:37.745319+0000","last_clean":"2024-02-10T07:38:37.745319+0000","last_became_active":"2024-02-10T07:38:02.184661+0000","last_became_peered":"2024-02-10T07:38:02.184661+0000","last_unstale":"2024-02-10T07:38:37.745319+0000","last_undegraded":"2024-02-10T07:38:37.745319+0000","last_fullsized":"2024-02-10T07:38:37.745319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:16:33.565149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834804+0000","last_change":"2024-02-10T07:37:58.148204+0000","last_active":"2024-02-10T07:38:37.834804+0000","last_peered":"2024-02-10T07:38:37.834804+0000","last_clean":"2024-02-10T07:38:37.834804+0000","last_became_active":"2024-02-10T07:37:58.147899+0000","last_became_peered":"2024-02-10T07:37:58.147899+0000","last_unstale":"2024-02-10T07:38:37.834804+0000","last_undegraded":"2024-02-10T07:38:37.834804+0000","last_fullsized":"2024-02-10T07:38:37.834804+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:14:58.792279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850587+0000","last_change":"2024-02-10T07:38:00.170125+0000","last_active":"2024-02-10T07:38:37.850587+0000","last_peered":"2024-02-10T07:38:37.850587+0000","last_clean":"2024-02-10T07:38:37.850587+0000","last_became_active":"2024-02-10T07:38:00.169801+0000","last_became_peered":"2024-02-10T07:38:00.169801+0000","last_unstale":"2024-02-10T07:38:37.850587+0000","last_undegraded":"2024-02-10T07:38:37.850587+0000","last_fullsized":"2024-02-10T07:38:37.850587+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:28:09.882257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850908+0000","last_change":"2024-02-10T07:38:04.194063+0000","last_active":"2024-02-10T07:38:37.850908+0000","last_peered":"2024-02-10T07:38:37.850908+0000","last_clean":"2024-02-10T07:38:37.850908+0000","last_became_active":"2024-02-10T07:38:04.193683+0000","last_became_peered":"2024-02-10T07:38:04.193683+0000","last_unstale":"2024-02-10T07:38:37.850908+0000","last_undegraded":"2024-02-10T07:38:37.850908+0000","last_fullsized":"2024-02-10T07:38:37.850908+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:10:59.476612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"65'2","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834438+0000","last_change":"2024-02-10T07:38:01.170203+0000","last_active":"2024-02-10T07:38:37.834438+0000","last_peered":"2024-02-10T07:38:37.834438+0000","last_clean":"2024-02-10T07:38:37.834438+0000","last_became_active":"2024-02-10T07:37:57.144810+0000","last_became_peered":"2024-02-10T07:37:57.144810+0000","last_unstale":"2024-02-10T07:38:37.834438+0000","last_undegraded":"2024-02-10T07:38:37.834438+0000","last_fullsized":"2024-02-10T07:38:37.834438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_clean_scrub_stamp":"2024-02-10T07:37:56.135799+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:19:56.935939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017851709999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748561+0000","last_change":"2024-02-10T07:37:58.159097+0000","last_active":"2024-02-10T07:38:37.748561+0000","last_peered":"2024-02-10T07:38:37.748561+0000","last_clean":"2024-02-10T07:38:37.748561+0000","last_became_active":"2024-02-10T07:37:58.158404+0000","last_became_peered":"2024-02-10T07:37:58.158404+0000","last_unstale":"2024-02-10T07:38:37.748561+0000","last_undegraded":"2024-02-10T07:38:37.748561+0000","last_fullsized":"2024-02-10T07:38:37.748561+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:20:55.223644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"69'28","reported_seq":93,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211514+0000","last_change":"2024-02-10T07:38:00.173241+0000","last_active":"2024-02-10T07:39:08.211514+0000","last_peered":"2024-02-10T07:39:08.211514+0000","last_clean":"2024-02-10T07:39:08.211514+0000","last_became_active":"2024-02-10T07:38:00.172634+0000","last_became_peered":"2024-02-10T07:38:00.172634+0000","last_unstale":"2024-02-10T07:39:08.211514+0000","last_undegraded":"2024-02-10T07:39:08.211514+0000","last_fullsized":"2024-02-10T07:39:08.211514+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:14:10.506150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848674+0000","last_change":"2024-02-10T07:38:02.191300+0000","last_active":"2024-02-10T07:38:37.848674+0000","last_peered":"2024-02-10T07:38:37.848674+0000","last_clean":"2024-02-10T07:38:37.848674+0000","last_became_active":"2024-02-10T07:38:02.190972+0000","last_became_peered":"2024-02-10T07:38:02.190972+0000","last_unstale":"2024-02-10T07:38:37.848674+0000","last_undegraded":"2024-02-10T07:38:37.848674+0000","last_fullsized":"2024-02-10T07:38:37.848674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:55:16.795143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844933+0000","last_change":"2024-02-10T07:38:04.561735+0000","last_active":"2024-02-10T07:38:37.844933+0000","last_peered":"2024-02-10T07:38:37.844933+0000","last_clean":"2024-02-10T07:38:37.844933+0000","last_became_active":"2024-02-10T07:38:04.561417+0000","last_became_peered":"2024-02-10T07:38:04.561417+0000","last_unstale":"2024-02-10T07:38:37.844933+0000","last_undegraded":"2024-02-10T07:38:37.844933+0000","last_fullsized":"2024-02-10T07:38:37.844933+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:55:04.134680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849450+0000","last_change":"2024-02-10T07:37:58.151796+0000","last_active":"2024-02-10T07:38:37.849450+0000","last_peered":"2024-02-10T07:38:37.849450+0000","last_clean":"2024-02-10T07:38:37.849450+0000","last_became_active":"2024-02-10T07:37:58.151501+0000","last_became_peered":"2024-02-10T07:37:58.151501+0000","last_unstale":"2024-02-10T07:38:37.849450+0000","last_undegraded":"2024-02-10T07:38:37.849450+0000","last_fullsized":"2024-02-10T07:38:37.849450+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:13:28.280284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.744807+0000","last_change":"2024-02-10T07:38:01.167119+0000","last_active":"2024-02-10T07:38:37.744807+0000","last_peered":"2024-02-10T07:38:37.744807+0000","last_clean":"2024-02-10T07:38:37.744807+0000","last_became_active":"2024-02-10T07:37:57.145944+0000","last_became_peered":"2024-02-10T07:37:57.145944+0000","last_unstale":"2024-02-10T07:38:37.744807+0000","last_undegraded":"2024-02-10T07:38:37.744807+0000","last_fullsized":"2024-02-10T07:38:37.744807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_clean_scrub_stamp":"2024-02-10T07:37:56.135799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:53:49.849478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050838299999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"69'13","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.746819+0000","last_change":"2024-02-10T07:38:28.792025+0000","last_active":"2024-02-10T07:38:37.746819+0000","last_peered":"2024-02-10T07:38:37.746819+0000","last_clean":"2024-02-10T07:38:37.746819+0000","last_became_active":"2024-02-10T07:38:28.776679+0000","last_became_peered":"2024-02-10T07:38:28.776679+0000","last_unstale":"2024-02-10T07:38:37.746819+0000","last_undegraded":"2024-02-10T07:38:37.746819+0000","last_fullsized":"2024-02-10T07:38:37.746819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:00:43.901323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745918+0000","last_change":"2024-02-10T07:38:02.184270+0000","last_active":"2024-02-10T07:38:37.745918+0000","last_peered":"2024-02-10T07:38:37.745918+0000","last_clean":"2024-02-10T07:38:37.745918+0000","last_became_active":"2024-02-10T07:38:02.183930+0000","last_became_peered":"2024-02-10T07:38:02.183930+0000","last_unstale":"2024-02-10T07:38:37.745918+0000","last_undegraded":"2024-02-10T07:38:37.745918+0000","last_fullsized":"2024-02-10T07:38:37.745918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:42:04.396161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834724+0000","last_change":"2024-02-10T07:38:04.193266+0000","last_active":"2024-02-10T07:38:37.834724+0000","last_peered":"2024-02-10T07:38:37.834724+0000","last_clean":"2024-02-10T07:38:37.834724+0000","last_became_active":"2024-02-10T07:38:04.192951+0000","last_became_peered":"2024-02-10T07:38:04.192951+0000","last_unstale":"2024-02-10T07:38:37.834724+0000","last_undegraded":"2024-02-10T07:38:37.834724+0000","last_fullsized":"2024-02-10T07:38:37.834724+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:54:15.160943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":278,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:39.832820+0000","last_change":"2024-02-10T07:37:47.705354+0000","last_active":"2024-02-10T07:38:39.832820+0000","last_peered":"2024-02-10T07:38:39.832820+0000","last_clean":"2024-02-10T07:38:39.832820+0000","last_became_active":"2024-02-10T07:37:47.704903+0000","last_became_peered":"2024-02-10T07:37:47.704903+0000","last_unstale":"2024-02-10T07:38:39.832820+0000","last_undegraded":"2024-02-10T07:38:39.832820+0000","last_fullsized":"2024-02-10T07:38:39.832820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:35:41.758084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:35:41.758084+0000","last_clean_scrub_stamp":"2024-02-10T07:35:41.758084+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:54:25.399467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848639+0000","last_change":"2024-02-10T07:37:58.151789+0000","last_active":"2024-02-10T07:38:37.848639+0000","last_peered":"2024-02-10T07:38:37.848639+0000","last_clean":"2024-02-10T07:38:37.848639+0000","last_became_active":"2024-02-10T07:37:58.151497+0000","last_became_peered":"2024-02-10T07:37:58.151497+0000","last_unstale":"2024-02-10T07:38:37.848639+0000","last_undegraded":"2024-02-10T07:38:37.848639+0000","last_fullsized":"2024-02-10T07:38:37.848639+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:33:00.081823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"69'16","reported_seq":67,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.212001+0000","last_change":"2024-02-10T07:38:00.767903+0000","last_active":"2024-02-10T07:39:08.212001+0000","last_peered":"2024-02-10T07:39:08.212001+0000","last_clean":"2024-02-10T07:39:08.212001+0000","last_became_active":"2024-02-10T07:38:00.767587+0000","last_became_peered":"2024-02-10T07:38:00.767587+0000","last_unstale":"2024-02-10T07:39:08.212001+0000","last_undegraded":"2024-02-10T07:39:08.212001+0000","last_fullsized":"2024-02-10T07:39:08.212001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:31:33.266260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844488+0000","last_change":"2024-02-10T07:38:02.188884+0000","last_active":"2024-02-10T07:38:37.844488+0000","last_peered":"2024-02-10T07:38:37.844488+0000","last_clean":"2024-02-10T07:38:37.844488+0000","last_became_active":"2024-02-10T07:38:02.188374+0000","last_became_peered":"2024-02-10T07:38:02.188374+0000","last_unstale":"2024-02-10T07:38:37.844488+0000","last_undegraded":"2024-02-10T07:38:37.844488+0000","last_fullsized":"2024-02-10T07:38:37.844488+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:28:53.296511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848833+0000","last_change":"2024-02-10T07:38:04.191260+0000","last_active":"2024-02-10T07:38:37.848833+0000","last_peered":"2024-02-10T07:38:37.848833+0000","last_clean":"2024-02-10T07:38:37.848833+0000","last_became_active":"2024-02-10T07:38:04.190966+0000","last_became_peered":"2024-02-10T07:38:04.190966+0000","last_unstale":"2024-02-10T07:38:37.848833+0000","last_undegraded":"2024-02-10T07:38:37.848833+0000","last_fullsized":"2024-02-10T07:38:37.848833+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:33:25.377619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848164+0000","last_change":"2024-02-10T07:37:58.157363+0000","last_active":"2024-02-10T07:38:37.848164+0000","last_peered":"2024-02-10T07:38:37.848164+0000","last_clean":"2024-02-10T07:38:37.848164+0000","last_became_active":"2024-02-10T07:37:58.157042+0000","last_became_peered":"2024-02-10T07:37:58.157042+0000","last_unstale":"2024-02-10T07:38:37.848164+0000","last_undegraded":"2024-02-10T07:38:37.848164+0000","last_fullsized":"2024-02-10T07:38:37.848164+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:18:17.067317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"69'5","reported_seq":113,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:12.023828+0000","last_change":"2024-02-10T07:38:01.169836+0000","last_active":"2024-02-10T07:39:12.023828+0000","last_peered":"2024-02-10T07:39:12.023828+0000","last_clean":"2024-02-10T07:39:12.023828+0000","last_became_active":"2024-02-10T07:37:57.147550+0000","last_became_peered":"2024-02-10T07:37:57.147550+0000","last_unstale":"2024-02-10T07:39:12.023828+0000","last_undegraded":"2024-02-10T07:39:12.023828+0000","last_fullsized":"2024-02-10T07:39:12.023828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:56.135799+0000","last_clean_scrub_stamp":"2024-02-10T07:37:56.135799+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:45:42.849143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090214500000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"69'12","reported_seq":47,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849359+0000","last_change":"2024-02-10T07:38:00.166353+0000","last_active":"2024-02-10T07:38:37.849359+0000","last_peered":"2024-02-10T07:38:37.849359+0000","last_clean":"2024-02-10T07:38:37.849359+0000","last_became_active":"2024-02-10T07:38:00.165935+0000","last_became_peered":"2024-02-10T07:38:00.165935+0000","last_unstale":"2024-02-10T07:38:37.849359+0000","last_undegraded":"2024-02-10T07:38:37.849359+0000","last_fullsized":"2024-02-10T07:38:37.849359+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:36:42.497334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.835018+0000","last_change":"2024-02-10T07:38:02.188399+0000","last_active":"2024-02-10T07:38:37.835018+0000","last_peered":"2024-02-10T07:38:37.835018+0000","last_clean":"2024-02-10T07:38:37.835018+0000","last_became_active":"2024-02-10T07:38:02.188087+0000","last_became_peered":"2024-02-10T07:38:02.188087+0000","last_unstale":"2024-02-10T07:38:37.835018+0000","last_undegraded":"2024-02-10T07:38:37.835018+0000","last_fullsized":"2024-02-10T07:38:37.835018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:39:28.135958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849012+0000","last_change":"2024-02-10T07:38:04.561979+0000","last_active":"2024-02-10T07:38:37.849012+0000","last_peered":"2024-02-10T07:38:37.849012+0000","last_clean":"2024-02-10T07:38:37.849012+0000","last_became_active":"2024-02-10T07:38:04.561422+0000","last_became_peered":"2024-02-10T07:38:04.561422+0000","last_unstale":"2024-02-10T07:38:37.849012+0000","last_undegraded":"2024-02-10T07:38:37.849012+0000","last_fullsized":"2024-02-10T07:38:37.849012+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:08:10.746152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849945+0000","last_change":"2024-02-10T07:37:58.156376+0000","last_active":"2024-02-10T07:38:37.849945+0000","last_peered":"2024-02-10T07:38:37.849945+0000","last_clean":"2024-02-10T07:38:37.849945+0000","last_became_active":"2024-02-10T07:37:58.156096+0000","last_became_peered":"2024-02-10T07:37:58.156096+0000","last_unstale":"2024-02-10T07:38:37.849945+0000","last_undegraded":"2024-02-10T07:38:37.849945+0000","last_fullsized":"2024-02-10T07:38:37.849945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:20:52.614395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"69'19","reported_seq":65,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850338+0000","last_change":"2024-02-10T07:38:00.765853+0000","last_active":"2024-02-10T07:38:37.850338+0000","last_peered":"2024-02-10T07:38:37.850338+0000","last_clean":"2024-02-10T07:38:37.850338+0000","last_became_active":"2024-02-10T07:38:00.765543+0000","last_became_peered":"2024-02-10T07:38:00.765543+0000","last_unstale":"2024-02-10T07:38:37.850338+0000","last_undegraded":"2024-02-10T07:38:37.850338+0000","last_fullsized":"2024-02-10T07:38:37.850338+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:35:14.838796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847870+0000","last_change":"2024-02-10T07:38:02.184373+0000","last_active":"2024-02-10T07:38:37.847870+0000","last_peered":"2024-02-10T07:38:37.847870+0000","last_clean":"2024-02-10T07:38:37.847870+0000","last_became_active":"2024-02-10T07:38:02.183764+0000","last_became_peered":"2024-02-10T07:38:02.183764+0000","last_unstale":"2024-02-10T07:38:37.847870+0000","last_undegraded":"2024-02-10T07:38:37.847870+0000","last_fullsized":"2024-02-10T07:38:37.847870+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:03:44.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847821+0000","last_change":"2024-02-10T07:38:04.199423+0000","last_active":"2024-02-10T07:38:37.847821+0000","last_peered":"2024-02-10T07:38:37.847821+0000","last_clean":"2024-02-10T07:38:37.847821+0000","last_became_active":"2024-02-10T07:38:04.199058+0000","last_became_peered":"2024-02-10T07:38:04.199058+0000","last_unstale":"2024-02-10T07:38:37.847821+0000","last_undegraded":"2024-02-10T07:38:37.847821+0000","last_fullsized":"2024-02-10T07:38:37.847821+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:28:11.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834994+0000","last_change":"2024-02-10T07:37:58.156606+0000","last_active":"2024-02-10T07:38:37.834994+0000","last_peered":"2024-02-10T07:38:37.834994+0000","last_clean":"2024-02-10T07:38:37.834994+0000","last_became_active":"2024-02-10T07:37:58.156297+0000","last_became_peered":"2024-02-10T07:37:58.156297+0000","last_unstale":"2024-02-10T07:38:37.834994+0000","last_undegraded":"2024-02-10T07:38:37.834994+0000","last_fullsized":"2024-02-10T07:38:37.834994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:20:21.511715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"69'10","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.745081+0000","last_change":"2024-02-10T07:38:28.781809+0000","last_active":"2024-02-10T07:38:37.745081+0000","last_peered":"2024-02-10T07:38:37.745081+0000","last_clean":"2024-02-10T07:38:37.745081+0000","last_became_active":"2024-02-10T07:38:28.776353+0000","last_became_peered":"2024-02-10T07:38:28.776353+0000","last_unstale":"2024-02-10T07:38:37.745081+0000","last_undegraded":"2024-02-10T07:38:37.745081+0000","last_fullsized":"2024-02-10T07:38:37.745081+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:35:00.739241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211491+0000","last_change":"2024-02-10T07:38:02.184061+0000","last_active":"2024-02-10T07:39:08.211491+0000","last_peered":"2024-02-10T07:39:08.211491+0000","last_clean":"2024-02-10T07:39:08.211491+0000","last_became_active":"2024-02-10T07:38:02.183267+0000","last_became_peered":"2024-02-10T07:38:02.183267+0000","last_unstale":"2024-02-10T07:39:08.211491+0000","last_undegraded":"2024-02-10T07:39:08.211491+0000","last_fullsized":"2024-02-10T07:39:08.211491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:16:15.305483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.844383+0000","last_change":"2024-02-10T07:38:04.563780+0000","last_active":"2024-02-10T07:38:37.844383+0000","last_peered":"2024-02-10T07:38:37.844383+0000","last_clean":"2024-02-10T07:38:37.844383+0000","last_became_active":"2024-02-10T07:38:04.563497+0000","last_became_peered":"2024-02-10T07:38:04.563497+0000","last_unstale":"2024-02-10T07:38:37.844383+0000","last_undegraded":"2024-02-10T07:38:37.844383+0000","last_fullsized":"2024-02-10T07:38:37.844383+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:53:01.554104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849492+0000","last_change":"2024-02-10T07:37:58.156387+0000","last_active":"2024-02-10T07:38:37.849492+0000","last_peered":"2024-02-10T07:38:37.849492+0000","last_clean":"2024-02-10T07:38:37.849492+0000","last_became_active":"2024-02-10T07:37:58.156090+0000","last_became_peered":"2024-02-10T07:37:58.156090+0000","last_unstale":"2024-02-10T07:38:37.849492+0000","last_undegraded":"2024-02-10T07:38:37.849492+0000","last_fullsized":"2024-02-10T07:38:37.849492+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:42:25.392619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"69'14","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747779+0000","last_change":"2024-02-10T07:38:00.160755+0000","last_active":"2024-02-10T07:38:37.747779+0000","last_peered":"2024-02-10T07:38:37.747779+0000","last_clean":"2024-02-10T07:38:37.747779+0000","last_became_active":"2024-02-10T07:38:00.160279+0000","last_became_peered":"2024-02-10T07:38:00.160279+0000","last_unstale":"2024-02-10T07:38:37.747779+0000","last_undegraded":"2024-02-10T07:38:37.747779+0000","last_fullsized":"2024-02-10T07:38:37.747779+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:37:03.981649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211526+0000","last_change":"2024-02-10T07:38:02.186969+0000","last_active":"2024-02-10T07:39:08.211526+0000","last_peered":"2024-02-10T07:39:08.211526+0000","last_clean":"2024-02-10T07:39:08.211526+0000","last_became_active":"2024-02-10T07:38:02.186668+0000","last_became_peered":"2024-02-10T07:38:02.186668+0000","last_unstale":"2024-02-10T07:39:08.211526+0000","last_undegraded":"2024-02-10T07:39:08.211526+0000","last_fullsized":"2024-02-10T07:39:08.211526+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:33:09.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747966+0000","last_change":"2024-02-10T07:38:04.193946+0000","last_active":"2024-02-10T07:38:37.747966+0000","last_peered":"2024-02-10T07:38:37.747966+0000","last_clean":"2024-02-10T07:38:37.747966+0000","last_became_active":"2024-02-10T07:38:04.193690+0000","last_became_peered":"2024-02-10T07:38:04.193690+0000","last_unstale":"2024-02-10T07:38:37.747966+0000","last_undegraded":"2024-02-10T07:38:37.747966+0000","last_fullsized":"2024-02-10T07:38:37.747966+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:45:47.474491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.851530+0000","last_change":"2024-02-10T07:37:58.152566+0000","last_active":"2024-02-10T07:38:37.851530+0000","last_peered":"2024-02-10T07:38:37.851530+0000","last_clean":"2024-02-10T07:38:37.851530+0000","last_became_active":"2024-02-10T07:37:58.152282+0000","last_became_peered":"2024-02-10T07:37:58.152282+0000","last_unstale":"2024-02-10T07:38:37.851530+0000","last_undegraded":"2024-02-10T07:38:37.851530+0000","last_fullsized":"2024-02-10T07:38:37.851530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:49:59.465503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"69'18","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848169+0000","last_change":"2024-02-10T07:38:00.765936+0000","last_active":"2024-02-10T07:38:37.848169+0000","last_peered":"2024-02-10T07:38:37.848169+0000","last_clean":"2024-02-10T07:38:37.848169+0000","last_became_active":"2024-02-10T07:38:00.765527+0000","last_became_peered":"2024-02-10T07:38:00.765527+0000","last_unstale":"2024-02-10T07:38:37.848169+0000","last_undegraded":"2024-02-10T07:38:37.848169+0000","last_fullsized":"2024-02-10T07:38:37.848169+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:37:57.750880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.749028+0000","last_change":"2024-02-10T07:38:02.186166+0000","last_active":"2024-02-10T07:38:37.749028+0000","last_peered":"2024-02-10T07:38:37.749028+0000","last_clean":"2024-02-10T07:38:37.749028+0000","last_became_active":"2024-02-10T07:38:02.185756+0000","last_became_peered":"2024-02-10T07:38:02.185756+0000","last_unstale":"2024-02-10T07:38:37.749028+0000","last_undegraded":"2024-02-10T07:38:37.749028+0000","last_fullsized":"2024-02-10T07:38:37.749028+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:09.439249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834707+0000","last_change":"2024-02-10T07:38:04.190094+0000","last_active":"2024-02-10T07:38:37.834707+0000","last_peered":"2024-02-10T07:38:37.834707+0000","last_clean":"2024-02-10T07:38:37.834707+0000","last_became_active":"2024-02-10T07:38:04.189824+0000","last_became_peered":"2024-02-10T07:38:04.189824+0000","last_unstale":"2024-02-10T07:38:37.834707+0000","last_undegraded":"2024-02-10T07:38:37.834707+0000","last_fullsized":"2024-02-10T07:38:37.834707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:40:06.349827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211889+0000","last_change":"2024-02-10T07:38:02.184785+0000","last_active":"2024-02-10T07:39:08.211889+0000","last_peered":"2024-02-10T07:39:08.211889+0000","last_clean":"2024-02-10T07:39:08.211889+0000","last_became_active":"2024-02-10T07:38:02.183131+0000","last_became_peered":"2024-02-10T07:38:02.183131+0000","last_unstale":"2024-02-10T07:39:08.211889+0000","last_undegraded":"2024-02-10T07:39:08.211889+0000","last_fullsized":"2024-02-10T07:39:08.211889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:39:22.937185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848084+0000","last_change":"2024-02-10T07:37:58.153227+0000","last_active":"2024-02-10T07:38:37.848084+0000","last_peered":"2024-02-10T07:38:37.848084+0000","last_clean":"2024-02-10T07:38:37.848084+0000","last_became_active":"2024-02-10T07:37:58.152937+0000","last_became_peered":"2024-02-10T07:37:58.152937+0000","last_unstale":"2024-02-10T07:38:37.848084+0000","last_undegraded":"2024-02-10T07:38:37.848084+0000","last_fullsized":"2024-02-10T07:38:37.848084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:55:00.998754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.848566+0000","last_change":"2024-02-10T07:38:00.170311+0000","last_active":"2024-02-10T07:38:37.848566+0000","last_peered":"2024-02-10T07:38:37.848566+0000","last_clean":"2024-02-10T07:38:37.848566+0000","last_became_active":"2024-02-10T07:38:00.169727+0000","last_became_peered":"2024-02-10T07:38:00.169727+0000","last_unstale":"2024-02-10T07:38:37.848566+0000","last_undegraded":"2024-02-10T07:38:37.848566+0000","last_fullsized":"2024-02-10T07:38:37.848566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:58:13.341066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850101+0000","last_change":"2024-02-10T07:38:04.562786+0000","last_active":"2024-02-10T07:38:37.850101+0000","last_peered":"2024-02-10T07:38:37.850101+0000","last_clean":"2024-02-10T07:38:37.850101+0000","last_became_active":"2024-02-10T07:38:04.561360+0000","last_became_peered":"2024-02-10T07:38:04.561360+0000","last_unstale":"2024-02-10T07:38:37.850101+0000","last_undegraded":"2024-02-10T07:38:37.850101+0000","last_fullsized":"2024-02-10T07:38:37.850101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:13.927336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.834918+0000","last_change":"2024-02-10T07:38:02.184847+0000","last_active":"2024-02-10T07:38:37.834918+0000","last_peered":"2024-02-10T07:38:37.834918+0000","last_clean":"2024-02-10T07:38:37.834918+0000","last_became_active":"2024-02-10T07:38:02.184558+0000","last_became_peered":"2024-02-10T07:38:02.184558+0000","last_unstale":"2024-02-10T07:38:37.834918+0000","last_undegraded":"2024-02-10T07:38:37.834918+0000","last_fullsized":"2024-02-10T07:38:37.834918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:34:57.087286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747625+0000","last_change":"2024-02-10T07:37:58.157306+0000","last_active":"2024-02-10T07:38:37.747625+0000","last_peered":"2024-02-10T07:38:37.747625+0000","last_clean":"2024-02-10T07:38:37.747625+0000","last_became_active":"2024-02-10T07:37:58.156985+0000","last_became_peered":"2024-02-10T07:37:58.156985+0000","last_unstale":"2024-02-10T07:38:37.747625+0000","last_undegraded":"2024-02-10T07:38:37.747625+0000","last_fullsized":"2024-02-10T07:38:37.747625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:21:32.639102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.750137+0000","last_change":"2024-02-10T07:38:00.162568+0000","last_active":"2024-02-10T07:38:37.750137+0000","last_peered":"2024-02-10T07:38:37.750137+0000","last_clean":"2024-02-10T07:38:37.750137+0000","last_became_active":"2024-02-10T07:38:00.162259+0000","last_became_peered":"2024-02-10T07:38:00.162259+0000","last_unstale":"2024-02-10T07:38:37.750137+0000","last_undegraded":"2024-02-10T07:38:37.750137+0000","last_fullsized":"2024-02-10T07:38:37.750137+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:52:51.018898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.747024+0000","last_change":"2024-02-10T07:38:04.200985+0000","last_active":"2024-02-10T07:38:37.747024+0000","last_peered":"2024-02-10T07:38:37.747024+0000","last_clean":"2024-02-10T07:38:37.747024+0000","last_became_active":"2024-02-10T07:38:04.200613+0000","last_became_peered":"2024-02-10T07:38:04.200613+0000","last_unstale":"2024-02-10T07:38:37.747024+0000","last_undegraded":"2024-02-10T07:38:37.747024+0000","last_fullsized":"2024-02-10T07:38:37.747024+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:43:21.024175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.850158+0000","last_change":"2024-02-10T07:38:02.182150+0000","last_active":"2024-02-10T07:38:37.850158+0000","last_peered":"2024-02-10T07:38:37.850158+0000","last_clean":"2024-02-10T07:38:37.850158+0000","last_became_active":"2024-02-10T07:38:02.181811+0000","last_became_peered":"2024-02-10T07:38:02.181811+0000","last_unstale":"2024-02-10T07:38:37.850158+0000","last_undegraded":"2024-02-10T07:38:37.850158+0000","last_fullsized":"2024-02-10T07:38:37.850158+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:05:07.917637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.847525+0000","last_change":"2024-02-10T07:37:58.157114+0000","last_active":"2024-02-10T07:38:37.847525+0000","last_peered":"2024-02-10T07:38:37.847525+0000","last_clean":"2024-02-10T07:38:37.847525+0000","last_became_active":"2024-02-10T07:37:58.156848+0000","last_became_peered":"2024-02-10T07:37:58.156848+0000","last_unstale":"2024-02-10T07:38:37.847525+0000","last_undegraded":"2024-02-10T07:38:37.847525+0000","last_fullsized":"2024-02-10T07:38:37.847525+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:33:39.433851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"69'17","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.749712+0000","last_change":"2024-02-10T07:38:00.168389+0000","last_active":"2024-02-10T07:38:37.749712+0000","last_peered":"2024-02-10T07:38:37.749712+0000","last_clean":"2024-02-10T07:38:37.749712+0000","last_became_active":"2024-02-10T07:38:00.168111+0000","last_became_peered":"2024-02-10T07:38:00.168111+0000","last_unstale":"2024-02-10T07:38:37.749712+0000","last_undegraded":"2024-02-10T07:38:37.749712+0000","last_fullsized":"2024-02-10T07:38:37.749712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:08:01.001880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.750431+0000","last_change":"2024-02-10T07:38:04.190427+0000","last_active":"2024-02-10T07:38:37.750431+0000","last_peered":"2024-02-10T07:38:37.750431+0000","last_clean":"2024-02-10T07:38:37.750431+0000","last_became_active":"2024-02-10T07:38:04.190138+0000","last_became_peered":"2024-02-10T07:38:04.190138+0000","last_unstale":"2024-02-10T07:38:37.750431+0000","last_undegraded":"2024-02-10T07:38:37.750431+0000","last_fullsized":"2024-02-10T07:38:37.750431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:03.170925+0000","last_clean_scrub_stamp":"2024-02-10T07:38:03.170925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:40:22.147921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:39:08.211498+0000","last_change":"2024-02-10T07:38:02.184972+0000","last_active":"2024-02-10T07:39:08.211498+0000","last_peered":"2024-02-10T07:39:08.211498+0000","last_clean":"2024-02-10T07:39:08.211498+0000","last_became_active":"2024-02-10T07:38:02.183251+0000","last_became_peered":"2024-02-10T07:38:02.183251+0000","last_unstale":"2024-02-10T07:39:08.211498+0000","last_undegraded":"2024-02-10T07:39:08.211498+0000","last_fullsized":"2024-02-10T07:39:08.211498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:38:01.157083+0000","last_clean_scrub_stamp":"2024-02-10T07:38:01.157083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:32:49.562659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.849142+0000","last_change":"2024-02-10T07:37:58.154971+0000","last_active":"2024-02-10T07:38:37.849142+0000","last_peered":"2024-02-10T07:38:37.849142+0000","last_clean":"2024-02-10T07:38:37.849142+0000","last_became_active":"2024-02-10T07:37:58.154664+0000","last_became_peered":"2024-02-10T07:37:58.154664+0000","last_unstale":"2024-02-10T07:38:37.849142+0000","last_undegraded":"2024-02-10T07:38:37.849142+0000","last_fullsized":"2024-02-10T07:38:37.849142+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:57.136924+0000","last_clean_scrub_stamp":"2024-02-10T07:37:57.136924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:08:09.230585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-02-10T07:38:37.748254+0000","last_change":"2024-02-10T07:38:00.164094+0000","last_active":"2024-02-10T07:38:37.748254+0000","last_peered":"2024-02-10T07:38:37.748254+0000","last_clean":"2024-02-10T07:38:37.748254+0000","last_became_active":"2024-02-10T07:38:00.163805+0000","last_became_peered":"2024-02-10T07:38:00.163805+0000","last_unstale":"2024-02-10T07:38:37.748254+0000","last_undegraded":"2024-02-10T07:38:37.748254+0000","last_fullsized":"2024-02-10T07:38:37.748254+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:37:59.144136+0000","last_clean_scrub_stamp":"2024-02-10T07:37:59.144136+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:01:35.721568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":63,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201300,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28304,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716848,"statfs":{"total":95995035648,"available":95966052352,"internally_reserved":0,"allocated":1318912,"data_stored":874659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.502}]},{"osd":1,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.434},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.47399999999999998}]},{"osd":3,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48499999999999999}]},{"osd":4,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.44600000000000001}]},{"osd":6,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51800000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462937,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32448,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712704,"statfs":{"total":95995035648,"available":95961808896,"internally_reserved":0,"allocated":1302528,"data_stored":873621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48399999999999999}]},{"osd":1,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":12.202999999999999,"5min":12.202999999999999,"15min":12.202999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45600000000000002}]},{"osd":2,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63}]},{"osd":3,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":12.118,"5min":12.118,"15min":12.118},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.64600000000000002}]},{"osd":5,"last update":"Sat Feb 10 07:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717396,"statfs":{"total":95995035648,"available":95966613504,"internally_reserved":0,"allocated":753664,"data_stored":284585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:28 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.80900000000000005,"15min":0.99099999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.58799999999999997,"5min":16.248999999999999,"15min":16.248999999999999},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.38500000000000001,"15min":0.36099999999999999},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59299999999999997,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40999999999999998},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.625,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32400000000000001}]},{"osd":3,"last update":"Sat Feb 10 07:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.378},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.496,"15min":0.499},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.39800000000000002}]},{"osd":4,"last update":"Sat Feb 10 07:38:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80800000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42999999999999999}]},{"osd":5,"last update":"Sat Feb 10 07:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75900000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63600000000000001,"15min":0.64500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76100000000000001,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48799999999999999}]},{"osd":6,"last update":"Sat Feb 10 07:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59099999999999997}]},{"osd":7,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":875674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 07:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.35599999999999998,"15min":0.34100000000000003},"min":{"1min":0.247,"5min":0.193,"15min":0.193},"max":{"1min":0.58399999999999996,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.371},"min":{"1min":0.33200000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.39900000000000002,"15min":0.39700000000000002},"min":{"1min":0.24399999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54400000000000004,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.42299999999999999,"15min":0.41099999999999998},"min":{"1min":0.29999999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.60499999999999998}]},{"osd":3,"last update":"Sat Feb 10 07:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.46100000000000002,"15min":0.47499999999999998},"min":{"1min":0.32800000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63400000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.44,"15min":0.45100000000000001},"min":{"1min":0.28499999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Feb 10 07:38:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.76200000000000001}]},{"osd":5,"last update":"Sat Feb 10 07:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.86499999999999999}]},{"osd":6,"last update":"Sat Feb 10 07:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.82499999999999996}]},{"osd":7,"last update":"Sat Feb 10 07:38:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.79100000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717428,"statfs":{"total":95995035648,"available":95966646272,"internally_reserved":0,"allocated":720896,"data_stored":284192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.372,"15min":0.376},"min":{"1min":0.26600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.51800000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.41499999999999998,"15min":0.40899999999999997},"min":{"1min":0.27900000000000003,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.27700000000000002}]},{"osd":1,"last update":"Sat Feb 10 07:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.41499999999999998,"15min":0.40500000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.28000000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.378}]},{"osd":3,"last update":"Sat Feb 10 07:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.49299999999999999,"15min":0.501},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.64500000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.47799999999999998,"15min":0.48499999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Feb 10 07:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58699999999999997}]},{"osd":5,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.58999999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.80600000000000005}]},{"osd":6,"last update":"Sat Feb 10 07:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sat Feb 10 07:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.753},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60099999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717400,"statfs":{"total":95995035648,"available":95966617600,"internally_reserved":0,"allocated":749568,"data_stored":285018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.45900000000000002,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.59899999999999998,"5min":5.1920000000000002,"15min":5.1920000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.224,"5min":0.223,"15min":0.223},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.36399999999999999}]},{"osd":1,"last update":"Sat Feb 10 07:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.442,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68000000000000005,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.313},{"interface":"front","average":{"1min":0.373,"5min":0.432,"15min":0.437},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65400000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.33800000000000002}]},{"osd":2,"last update":"Sat Feb 10 07:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.44700000000000001,"15min":0.45700000000000002},"min":{"1min":0.28100000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.69899999999999995,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.50900000000000001,"15min":0.46999999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70999999999999996,"5min":5.1470000000000002,"15min":5.1470000000000002},"last":0.26500000000000001}]},{"osd":4,"last update":"Sat Feb 10 07:38:31 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.623,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":5.1180000000000003,"5min":5.1180000000000003,"15min":5.1180000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.64500000000000002,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.3819999999999997,"5min":7.3819999999999997,"15min":7.3819999999999997},"last":0.43099999999999999}]},{"osd":5,"last update":"Sat Feb 10 07:38:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.68799999999999994,"15min":0.66000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.077,"5min":5.077,"15min":5.077},"last":0.41999999999999998}]},{"osd":6,"last update":"Sat Feb 10 07:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":0.499},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.2210000000000001,"5min":5.2210000000000001,"15min":5.2210000000000001},"last":0.52000000000000002}]},{"osd":7,"last update":"Sat Feb 10 07:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.45700000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":283845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.69299999999999995}]},{"osd":1,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":1.2529999999999999,"15min":1.3819999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":18.132999999999999,"15min":18.132999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.76600000000000001}]},{"osd":3,"last update":"Sat Feb 10 07:38:32 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":1.2829999999999999,"15min":1.411},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.64600000000000002,"5min":18.106999999999999,"15min":18.106999999999999},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.39100000000000001},"min":{"1min":0.27300000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.745}]},{"osd":6,"last update":"Sat Feb 10 07:38:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Feb 10 07:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.64800000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724574,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713268,"statfs":{"total":95995035648,"available":95962386432,"internally_reserved":0,"allocated":720896,"data_stored":284850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68500000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.55600000000000005,"15min":0.54500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68400000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.55300000000000005}]},{"osd":2,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73099999999999998,"5min":0.746,"15min":0.746},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.40000000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.77300000000000002}]},{"osd":4,"last update":"Sat Feb 10 07:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"min":{"1min":0.309,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.755}]},{"osd":6,"last update":"Sat Feb 10 07:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.70499999999999996}]},{"osd":7,"last update":"Sat Feb 10 07:38:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T07:39:18.618 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-10T07:39:18.618 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-10T07:39:18.619 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-10T07:39:18.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 -- ceph health --format=json 2024-02-10T07:39:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:18 smithi155 ceph-mon[32688]: from='client.34649 v1:172.21.15.70:0/2026428545' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:39:19.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:39:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:39:19.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:19 smithi070 ceph-mon[39923]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:19.561 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:19 smithi070 ceph-mon[29548]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:19 smithi155 ceph-mon[32688]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:20.538 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:20.538 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:21.192 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:39:21.587 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:21 smithi070 ceph-mon[39923]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:21.588 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:21 smithi070 ceph-mon[29548]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:21 smithi155 ceph-mon[32688]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:22.188 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:39:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:39:23.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:23 smithi070 ceph-mon[39923]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:23.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:23 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:23 smithi070 ceph-mon[29548]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:23 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:23.492 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:23 smithi070 podman[75268]: 2024-02-10 07:39:23.405102173 +0000 UTC m=+0.454458202 container create f8d8f1bec1f5bc792b1d5567c658270662d6acaed980557d6c8a8dda8476e024 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:23 smithi155 ceph-mon[32688]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:23 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:24.438 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 podman[75268]: 2024-02-10 07:39:24.21177997 +0000 UTC m=+1.261135999 container init f8d8f1bec1f5bc792b1d5567c658270662d6acaed980557d6c8a8dda8476e024 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 podman[75268]: 2024-02-10 07:39:24.215150921 +0000 UTC m=+1.264506957 container start f8d8f1bec1f5bc792b1d5567c658270662d6acaed980557d6c8a8dda8476e024 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.229Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.229Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.230Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.70 port=9094 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.232Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.257Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.257Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.260Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:24.260Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 bash[75268]: f8d8f1bec1f5bc792b1d5567c658270662d6acaed980557d6c8a8dda8476e024 2024-02-10T07:39:24.439 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:24 smithi070 systemd[1]: Started Ceph alertmanager.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:39:24.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.542+0000 7f821a9bc700 1 -- 172.21.15.70:0/2939547316 >> v1:172.21.15.70:6789/0 conn(0x7f821410aa80 legacy=0x7f821410ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:24.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.542+0000 7f821a9bc700 1 -- 172.21.15.70:0/2939547316 shutdown_connections 2024-02-10T07:39:24.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.542+0000 7f821a9bc700 1 -- 172.21.15.70:0/2939547316 >> 172.21.15.70:0/2939547316 conn(0x7f82140fc4c0 msgr2=0x7f82140fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:24.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.542+0000 7f821a9bc700 1 -- 172.21.15.70:0/2939547316 shutdown_connections 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.542+0000 7f821a9bc700 1 -- 172.21.15.70:0/2939547316 wait complete. 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.542+0000 7f821a9bc700 1 Processor -- start 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f821a9bc700 1 -- start start 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f821a9bc700 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f82141a8b80 con 0x7f821410aa80 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f821a9bc700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f82141a9d20 con 0x7f821410e590 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f821a9bc700 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f82141aaf20 con 0x7f8214102c70 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f82137fe700 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f821410aa80 0x7f82141a3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57028/0 (socket says 172.21.15.70:57028) 2024-02-10T07:39:24.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f82137fe700 1 -- 172.21.15.70:0/3321962185 learned_addr learned my addr 172.21.15.70:0/3321962185 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T07:39:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014207413 0 0) 0x7f82141aaf20 con 0x7f8214102c70 2024-02-10T07:39:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81fc0035b0 con 0x7f8214102c70 2024-02-10T07:39:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129271291 0 0) 0x7f82141a8b80 con 0x7f821410aa80 2024-02-10T07:39:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82141aaf20 con 0x7f821410aa80 2024-02-10T07:39:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.543+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195845381 0 0) 0x7f82141a9d20 con 0x7f821410e590 2024-02-10T07:39:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82141a8b80 con 0x7f821410e590 2024-02-10T07:39:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3726369182 0 0) 0x7f81fc0035b0 con 0x7f8214102c70 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82141a9d20 con 0x7f8214102c70 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f82040018b0 con 0x7f8214102c70 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2072358255 0 0) 0x7f82141aaf20 con 0x7f821410aa80 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81fc0035b0 con 0x7f821410aa80 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8208004470 con 0x7f821410aa80 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3619330976 0 0) 0x7f82141a8b80 con 0x7f821410e590 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82141aaf20 con 0x7f821410e590 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f820c001bb0 con 0x7f821410e590 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 593910559 0 0) 0x7f82141a9d20 con 0x7f8214102c70 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 >> v1:172.21.15.155:6789/0 conn(0x7f821410e590 legacy=0x7f82141a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 >> v1:172.21.15.70:6789/0 conn(0x7f821410aa80 legacy=0x7f82141a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82141ac120 con 0x7f8214102c70 2024-02-10T07:39:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82141a8d70 con 0x7f8214102c70 2024-02-10T07:39:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a91f0 con 0x7f8214102c70 2024-02-10T07:39:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f8204003b30 con 0x7f8214102c70 2024-02-10T07:39:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.544+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8204004ac0 con 0x7f8214102c70 2024-02-10T07:39:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.545+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8200005120 con 0x7f8214102c70 2024-02-10T07:39:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.545+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f820401b150 con 0x7f8214102c70 2024-02-10T07:39:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.546+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 4216839452 0 0) 0x7f82040877d0 con 0x7f8214102c70 2024-02-10T07:39:24.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.551+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8204054930 con 0x7f8214102c70 2024-02-10T07:39:24.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:39:24] ENGINE Bus STOPPING 2024-02-10T07:39:24.741 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:39:24] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:39:24.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:39:24] ENGINE Bus STOPPED 2024-02-10T07:39:24.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:39:24] ENGINE Bus STARTING 2024-02-10T07:39:24.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:39:24] ENGINE Serving on http://:::9283 2024-02-10T07:39:24.742 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:24 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:39:24] ENGINE Bus STARTED 2024-02-10T07:39:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.790+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8200005950 con 0x7f8214102c70 2024-02-10T07:39:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.790+0000 7f82117fa700 1 -- 172.21.15.70:0/3321962185 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f8204052f30 con 0x7f8214102c70 2024-02-10T07:39:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-02-10T07:39:24.792 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-10T07:39:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.792+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 >> v1:172.21.15.70:6800/4101821536 conn(0x7f81fc06eb80 legacy=0x7f81fc071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.792+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 >> v1:172.21.15.70:6790/0 conn(0x7f8214102c70 legacy=0x7f821410a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T07:39:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.793+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 shutdown_connections 2024-02-10T07:39:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.793+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 >> 172.21.15.70:0/3321962185 conn(0x7f82140fc4c0 msgr2=0x7f82141109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T07:39:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.793+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 shutdown_connections 2024-02-10T07:39:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:2024-02-10T07:39:24.793+0000 7f821a9bc700 1 -- 172.21.15.70:0/3321962185 wait complete. 2024-02-10T07:39:25.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:25.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T07:39:25.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:39:25.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:39:25.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3321962185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T07:39:25.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:25.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.428 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.429 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T07:39:25.429 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.429 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.429 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:39:25.429 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:39:25.429 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.430 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3321962185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T07:39:25.454 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-10T07:39:25.454 INFO:tasks.cephadm:Setup complete, yielding 2024-02-10T07:39:25.454 INFO:teuthology.run_tasks:Running task workunit... 2024-02-10T07:39:25.467 INFO:tasks.workunit:Pulling workunits from ref 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:39:25.468 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-10T07:39:25.468 DEBUG:teuthology.orchestra.run.smithi070:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-10T07:39:25.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:39:25.484 INFO:teuthology.orchestra.run.smithi070.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-10T07:39:25.484 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-10T07:39:25.539 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-10T07:39:25.539 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-10T07:39:25.594 INFO:tasks.workunit:timeout=3h 2024-02-10T07:39:25.594 INFO:tasks.workunit:cleanup=True 2024-02-10T07:39:25.594 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:39:25.650 INFO:tasks.workunit.client.0.smithi070.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:39:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:39:25.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:25.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3321962185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T07:39:26.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:26 smithi070 ceph-mon[29548]: Deploying daemon grafana.a on smithi155 2024-02-10T07:39:26.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:26 smithi070 ceph-mon[39923]: Deploying daemon grafana.a on smithi155 2024-02-10T07:39:26.491 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:26 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:26.232Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000150895s 2024-02-10T07:39:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:26 smithi155 ceph-mon[32688]: Deploying daemon grafana.a on smithi155 2024-02-10T07:39:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:27 smithi070 ceph-mon[29548]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:27.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:27 smithi070 ceph-mon[39923]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:27 smithi155 ceph-mon[32688]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:29.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:28 smithi070 ceph-mon[29548]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:29.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:28 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:29.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:28 smithi070 ceph-mon[39923]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:29.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:28 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:29.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:39:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:39:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:28 smithi155 ceph-mon[32688]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:28 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:29 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:30.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:29 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:29 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:31.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:30 smithi070 ceph-mon[29548]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:31.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:30 smithi070 ceph-mon[39923]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:30 smithi155 ceph-mon[32688]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:39:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:39:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:32 smithi070 ceph-mon[29548]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:33.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:32 smithi070 ceph-mon[39923]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:33.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:32 smithi155 ceph-mon[32688]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:34.491 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:34 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:34.234Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002033971s 2024-02-10T07:39:35.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:34 smithi070 ceph-mon[29548]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:35.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:34 smithi070 ceph-mon[39923]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:34 smithi155 ceph-mon[32688]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:37.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:36 smithi070 ceph-mon[29548]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:37.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:36 smithi070 ceph-mon[39923]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:37.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:36 smithi155 ceph-mon[32688]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:38.999 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:38 smithi155 ceph-mon[32688]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:38.999 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:38.999 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:39:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:39:39.000 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:38 smithi155 systemd[1]: Starting Ceph grafana.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:39:39.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:38 smithi070 ceph-mon[29548]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:39.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:39.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:38 smithi070 ceph-mon[39923]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:39.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:39.255 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 podman[63371]: 2024-02-10 07:39:39.065482495 +0000 UTC m=+0.016500087 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-10T07:39:39.255 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 podman[63371]: 2024-02-10 07:39:39.171458368 +0000 UTC m=+0.122475995 container create 9da082eb524410417420f495ee35da41764fc3679748ec0b17c279708d1a5119 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, vendor=Red Hat, Inc., io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0, name=ubi8, description=Ceph Grafana Container, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, com.redhat.component=ubi8-container, release=239.1651231664) 2024-02-10T07:39:39.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 podman[63371]: 2024-02-10 07:39:39.415706648 +0000 UTC m=+0.366724231 container init 9da082eb524410417420f495ee35da41764fc3679748ec0b17c279708d1a5119 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a, io.openshift.tags=base rhel8, name=ubi8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, distribution-scope=public, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., io.openshift.expose-services=, release=239.1651231664, maintainer=Paul Cuzner , io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, architecture=x86_64, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git) 2024-02-10T07:39:39.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 podman[63371]: 2024-02-10 07:39:39.418651006 +0000 UTC m=+0.369668597 container start 9da082eb524410417420f495ee35da41764fc3679748ec0b17c279708d1a5119 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, distribution-scope=public, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, release=239.1651231664, version=8.5, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, io.buildah.version=1.32.0, description=Ceph Grafana Container, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-02-10T07:39:39.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 bash[63371]: 9da082eb524410417420f495ee35da41764fc3679748ec0b17c279708d1a5119 2024-02-10T07:39:39.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 systemd[1]: Started Ceph grafana.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:39:39.824 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574054644Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-10T07:39:39.824 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574293692Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-10T07:39:39.824 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574320719Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.57434035Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574347902Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574354097Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574359718Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574365508Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574372399Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574378147Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574384087Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574389551Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.57439517Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=settings t=2024-02-10T07:39:39.574401368Z level=info msg="App mode production" 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=sqlstore t=2024-02-10T07:39:39.574457268Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=sqlstore t=2024-02-10T07:39:39.574489648Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-10T07:39:39.825 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.575511875Z level=info msg="Starting DB migrations" 2024-02-10T07:39:39.826 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.575591223Z level=info msg="Executing migration" id="create migration_log table" 2024-02-10T07:39:39.826 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.576115043Z level=info msg="Executing migration" id="create user table" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.576564896Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.577391312Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.578238696Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.579074305Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.579949145Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.580869565Z level=info msg="Executing migration" id="create user table v2" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.581666042Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.582539535Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.583421352Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.584236137Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-10T07:39:39.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.585086586Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.585918358Z level=info msg="Executing migration" id="Update user table charset" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.586792994Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.587254924Z level=info msg="Executing migration" id="Add missing user data" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.590114286Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.592982046Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.595802118Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.596265677Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.599039538Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.601829293Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.60470185Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.607477838Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.610006238Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-10T07:39:39.828 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.612849853Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.615603971Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.616014523Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.618748379Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.619175644Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.621553726Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.622233643Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.624305805Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.624736159Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.627422263Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.630052215Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.632520618Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.63287757Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.635545996Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-10T07:39:39.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.63590889Z level=info msg="Executing migration" id="create star table" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.638253069Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.638715998Z level=info msg="Executing migration" id="create org table v1" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.640879107Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.643590666Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.64606376Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.648718194Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.651366866Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.654068004Z level=info msg="Executing migration" id="Update org table charset" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.656654014Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.659322657Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.661894937Z level=info msg="Executing migration" id="create dashboard table" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.662769938Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-10T07:39:39.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.663588632Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.664432219Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.665320948Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.666136026Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.666984461Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.66786708Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.668639686Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.669439721Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.670258214Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.671113679Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.671968644Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.672762415Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.67358649Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.674418919Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.675229184Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.676057556Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-10T07:39:39.831 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.676896348Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.677721415Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.678571294Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.679491746Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.6803355Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.681210007Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.682059239Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.682909824Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.683670387Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.684520851Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.685394275Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.686248272Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.687080076Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-10T07:39:39.832 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.68796735Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.688836921Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.689759021Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.690601849Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.691389455Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.694280494Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.697091033Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.699906222Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.702421258Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.70511314Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.707882427Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.710692474Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.713462459Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.716172061Z level=info msg="Executing migration" id="create data_source table" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.719064432Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-10T07:39:39.833 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.721884389Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-10T07:39:39.834 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.724653264Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.727432929Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.730151398Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.732918746Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.735713776Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.738433549Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.741012203Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.743836132Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.746583804Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.749386086Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.752155731Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.754880387Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-10T07:39:39.835 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.757595803Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.758043466Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.760617116Z level=info msg="Executing migration" id="Add uid column" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.76118211Z level=info msg="Executing migration" id="Update uid value" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.763252901Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.763706819Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.765921117Z level=info msg="Executing migration" id="create api_key table" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.768568594Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.771010971Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.7735488Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.776102281Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.778646284Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.781140937Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.783621891Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-10T07:39:39.836 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.784679081Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.787108693Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.789593725Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.792044635Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.792475207Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.794901581Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.795295147Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.79767724Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.798222467Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.800701852Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.803208992Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.803928813Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.804499441Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-10T07:39:39.837 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.805232822Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.805935148Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.806340381Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.806766838Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.807183857Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.807922136Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.808615519Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.809333503Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.809905524Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.810656893Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.811365167Z level=info msg="Executing migration" id="create quota table v1" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.812088518Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.812518077Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.813202724Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.813589066Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-10T07:39:39.838 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.814332585Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.814936477Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.815687129Z level=info msg="Executing migration" id="create session table" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.816176063Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.816491436Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.816831625Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.817218929Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.817986488Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.818735751Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.81902514Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.819288503Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.819582083Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.81997846Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.820234292Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-10T07:39:39.839 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.820837396Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.82158912Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.82220079Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.82280269Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.823548276Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.823964709Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.824779931Z level=info msg="Executing migration" id="create alert table v1" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.825382948Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.826132464Z level=info msg="Executing migration" id="add index alert state" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.82688342Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.827311732Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.827744276Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.828179824Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.828642069Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.830159248Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-10T07:39:39.840 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.83056467Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-10T07:39:40.074 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:40.075 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.075 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.075 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.075 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.831061728Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.831389589Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.831791346Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.832228499Z level=info msg="Executing migration" id="Add column is_default" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.832963646Z level=info msg="Executing migration" id="Add column frequency" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.833628753Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.834275957Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.835054882Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.835586657Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.835925132Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.836240342Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-10T07:39:40.076 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.836704076Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.837294591Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.83776174Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.838205985Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.838659667Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.839301864Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.839949389Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.840271439Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.840727408Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.841158818Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.841832806Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.84214094Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.842557221Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.842998431Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-10T07:39:40.077 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.843304409Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.843754331Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.844499037Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.844940017Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.845394177Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.845920559Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.84666518Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.847327903Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.848005054Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.848418557Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.849206447Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.851949416Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.854699148Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.855136368Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-10T07:39:40.078 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.857872791Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.860602475Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.863323128Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.863706768Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.866500886Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.869078253Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.87188234Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.872612588Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.875274235Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.877944153Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.880392079Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.881100364Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.883757118Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.886413437Z level=info msg="Executing migration" id="Move region to single row" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.889029677Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-10T07:39:40.079 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.889475097Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-10T07:39:40.080 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.891955698Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-10T07:39:40.080 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.894393603Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.897065662Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.897505945Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.900308028Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.903128865Z level=info msg="Executing migration" id="create test_data table" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.905912667Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.906553147Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.909365544Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.912081121Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.914855618Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.917607999Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.920317246Z level=info msg="Executing migration" id="create team table" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.923051059Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.925573447Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.926021649Z level=info msg="Executing migration" id="create team member table" 2024-02-10T07:39:40.081 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.928735504Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.931539351Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.934244291Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.934699209Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.937355512Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.940034307Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.942627839Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.94305906Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.945595114Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.946466132Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.947275792Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.947762099Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.948556333Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.949346488Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.950188332Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-10T07:39:40.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.95103544Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.951822789Z level=info msg="Executing migration" id="create tag table" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.95261169Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.953431919Z level=info msg="Executing migration" id="create login attempt table" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.954274868Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.955093333Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.955892382Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.957783482Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.9586155Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.959509906Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.960350313Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.96119187Z level=info msg="Executing migration" id="create user auth table" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.962065888Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.962963658Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.963820542Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-10T07:39:40.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.964754769Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.965605308Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.966450947Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.967318523Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.968104842Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.968982672Z level=info msg="Executing migration" id="create server_lock table" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.96987776Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.970756144Z level=info msg="Executing migration" id="create user auth token table" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.971572906Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.972414569Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.973295839Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.974158738Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.974966252Z level=info msg="Executing migration" id="create cache_data table" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.975838911Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.978698771Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-10T07:39:40.084 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.98159758Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.984418848Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.987264589Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.989993646Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.992764369Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.995588476Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:39.998392043Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.001055295Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.003906181Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.006721012Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.009614496Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.012457877Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.015247158Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.017969915Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-10T07:39:40.085 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.020624381Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.02349556Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.026114343Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.028767142Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.03151242Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.034003959Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.036770406Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.0393884Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.042208402Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.044678769Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.047454115Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.050233142Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.053673382Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.057051724Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-10T07:39:40.086 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.059777111Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.062328269Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.065051907Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.067930527Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.070435519Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.073085515Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.073866986Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.074657425Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.075505977Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.076327764Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.077138735Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.077927705Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-10T07:39:40.087 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.078716835Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-10T07:39:40.240 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:40.240 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.240 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.240 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.240 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.240 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:40.329 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.079570917Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-10T07:39:40.329 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.080838224Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.081671191Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.082904727Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.084138423Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.084927786Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.08573628Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.08656366Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.087022758Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.087782407Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.088757869Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.089587116Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.090430346Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.091274107Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.092158217Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-10T07:39:40.330 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.09300328Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.093995299Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.094996946Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.095966904Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.097105021Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.09796402Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.098891031Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.099728108Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.100576003Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.10145842Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.101893053Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.102715646Z level=info msg="Executing migration" id="create alert_image table" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.10369425Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.106396861Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-10T07:39:40.331 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.106840151Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-10T07:39:40.332 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.10726889Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-10T07:39:40.332 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.109706466Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T07:39:40.332 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.109795034Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T07:39:40.332 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.112316702Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-10T07:39:40.332 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.112837795Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.115343948Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.115497023Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.115620625Z level=info msg="alerts found to migrate" alerts=0 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.116042875Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.118125527Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.12042871Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.120878524Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-10T07:39:40.333 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.123375947Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.125889559Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.128406944Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.128912885Z level=info msg="Executing migration" id="create data_keys table" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.12973511Z level=info msg="Executing migration" id="create secrets table" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.130711913Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.135258244Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.136176718Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.137067105Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.14143479Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.145653237Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.146582757Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.14721243Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.147987114Z level=info msg="Executing migration" id="create permission table" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.148412297Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-10T07:39:40.334 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.148881284Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.149669954Z level=info msg="Executing migration" id="create role table" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.150439143Z level=info msg="Executing migration" id="add column display_name" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.151374439Z level=info msg="Executing migration" id="add column group_name" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.152345669Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.153119977Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.15404761Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.154706267Z level=info msg="Executing migration" id="create team role table" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.155626184Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.156280426Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.15700298Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.157798503Z level=info msg="Executing migration" id="create user role table" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.158666272Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.159589921Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-10T07:39:40.335 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.160240615Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.161152708Z level=info msg="Executing migration" id="create builtin role table" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.161817722Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.162723838Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.163364661Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.164324643Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.164833145Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.165529114Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.165961798Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.166408543Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.166872325Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.167363214Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.168308598Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-10T07:39:40.336 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.168770537Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.169245413Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.16952726Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.16997402Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.170422013Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.170829604Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.171142532Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.171501162Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.171919232Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.172388207Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.173336991Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.173677165Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.174102033Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.174598694Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-10T07:39:40.337 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.175039129Z level=info msg="Executing migration" id="add correlation config column" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.176065275Z level=info msg="Executing migration" id="create entity_events table" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.176511106Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.177162709Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.177240477Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.177553245Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.177652623Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.177942717Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.178337499Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.178753616Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.179190727Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.179625056Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.180035229Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.180429454Z level=info msg="Executing migration" id="Drop public config table" 2024-02-10T07:39:40.338 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.180845551Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.18123332Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.181670961Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.182126488Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.184742982Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.187706392Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.188711885Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.189744043Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.190153127Z level=info msg="Executing migration" id="add share column" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.19118878Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.191631426Z level=info msg="Executing migration" id="create file table" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.192219466Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.194940666Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.195712091Z level=info msg="Executing migration" id="create file_meta table" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.198675303Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-10T07:39:40.339 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.201264187Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.203996662Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.20659801Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.209378792Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.212084651Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.214680858Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.217502677Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.220285321Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.222893194Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.225682275Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.22813727Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.230787373Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.233299694Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.23604723Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-10T07:39:40.340 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.245754876Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.248332474Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.250888459Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.254436431Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.257144323Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.259706411Z level=info msg="Executing migration" id="create folder table" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.262090859Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.264410931Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.266922118Z level=info msg="Executing migration" id="Update folder title length" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.26969623Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=migrator t=2024-02-10T07:39:40.272362592Z level=info msg="migrations completed" performed=485 skipped=0 duration=696.790922ms 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=sqlstore t=2024-02-10T07:39:40.272799597Z level=info msg="Created default organization" 2024-02-10T07:39:40.341 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=plugin.loader t=2024-02-10T07:39:40.329505132Z level=info msg="Plugin registered" pluginID=input 2024-02-10T07:39:40.583 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=plugin.loader t=2024-02-10T07:39:40.363109091Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-10T07:39:40.583 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=plugin.loader t=2024-02-10T07:39:40.363145787Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-10T07:39:40.583 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=secrets t=2024-02-10T07:39:40.363188248Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-10T07:39:40.583 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=query_data t=2024-02-10T07:39:40.365515958Z level=info msg="Query Service initialization" 2024-02-10T07:39:40.583 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=live.push_http t=2024-02-10T07:39:40.367904599Z level=info msg="Live Push Gateway initialization" 2024-02-10T07:39:40.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-mon[32688]: Deploying daemon node-exporter.a on smithi070 2024-02-10T07:39:40.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-mon[32688]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:40 smithi070 ceph-mon[29548]: Deploying daemon node-exporter.a on smithi070 2024-02-10T07:39:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:40 smithi070 ceph-mon[29548]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:40 smithi070 ceph-mon[39923]: Deploying daemon node-exporter.a on smithi070 2024-02-10T07:39:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:40 smithi070 ceph-mon[39923]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:40.992 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:40 smithi070 bash[75654]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T07:39:41.255 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=infra.usagestats.collector t=2024-02-10T07:39:40.989064292Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-10T07:39:41.256 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=provisioning.datasources t=2024-02-10T07:39:40.990171457Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-10T07:39:41.256 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=provisioning.datasources t=2024-02-10T07:39:41.001224883Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-10T07:39:41.256 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=provisioning.alerting t=2024-02-10T07:39:41.012050728Z level=info msg="starting to provision alerting" 2024-02-10T07:39:41.256 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=provisioning.alerting t=2024-02-10T07:39:41.012087391Z level=info msg="finished to provision alerting" 2024-02-10T07:39:41.257 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=grafanaStorageLogger t=2024-02-10T07:39:41.012248674Z level=info msg="storage starting" 2024-02-10T07:39:41.257 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=ngalert.state.manager t=2024-02-10T07:39:41.012262549Z level=info msg="Warming state cache for startup" 2024-02-10T07:39:41.257 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=http.server t=2024-02-10T07:39:41.014117075Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-10T07:39:41.257 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=ngalert.state.manager t=2024-02-10T07:39:41.018088297Z level=info msg="State cache has been initialized" states=0 duration=5.824888ms 2024-02-10T07:39:41.257 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=ticker t=2024-02-10T07:39:41.018154741Z level=info msg=starting first_tick=2024-02-10T07:39:50Z 2024-02-10T07:39:41.258 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=ngalert.multiorg.alertmanager t=2024-02-10T07:39:41.018156712Z level=info msg="starting MultiOrg Alertmanager" 2024-02-10T07:39:41.258 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:39:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=sqlstore.transactions t=2024-02-10T07:39:41.031978406Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-10T07:39:41.491 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:41 smithi070 bash[75654]: Getting image source signatures 2024-02-10T07:39:41.491 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:41 smithi070 bash[75654]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T07:39:41.491 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:41 smithi070 bash[75654]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T07:39:41.491 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:41 smithi070 bash[75654]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T07:39:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:39:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:39:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:42 smithi070 ceph-mon[29548]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:43.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:42 smithi070 ceph-mon[39923]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:43.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:42 smithi155 ceph-mon[32688]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:45.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:44 smithi070 ceph-mon[39923]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:44 smithi070 ceph-mon[29548]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:44 smithi155 ceph-mon[32688]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:46.241 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:45 smithi070 bash[75654]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T07:39:46.664 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 bash[75654]: Writing manifest to image destination 2024-02-10T07:39:46.664 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 podman[75654]: 2024-02-10 07:39:46.566654388 +0000 UTC m=+5.823496400 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T07:39:46.664 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 podman[75654]: 2024-02-10 07:39:46.664457735 +0000 UTC m=+5.921299726 container create 1ae0911b3151117fe5a9eda85e9f2554fb3c44292969f82c849d7ef3c1edb11f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:46.974 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-mon[39923]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:46.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-mon[29548]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:46.975 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 podman[75654]: 2024-02-10 07:39:46.866696226 +0000 UTC m=+6.123538223 container init 1ae0911b3151117fe5a9eda85e9f2554fb3c44292969f82c849d7ef3c1edb11f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:46.975 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 podman[75654]: 2024-02-10 07:39:46.873514786 +0000 UTC m=+6.130356776 container start 1ae0911b3151117fe5a9eda85e9f2554fb3c44292969f82c849d7ef3c1edb11f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:46.975 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.878Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T07:39:46.975 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.878Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.879Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.879Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T07:39:46.976 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T07:39:46.977 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T07:39:46.978 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T07:39:46.979 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T07:39:46.980 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T07:39:46.980 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.880Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T07:39:46.980 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.881Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T07:39:46.980 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a[75706]: ts=2024-02-10T07:39:46.881Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T07:39:47.241 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 bash[75654]: 1ae0911b3151117fe5a9eda85e9f2554fb3c44292969f82c849d7ef3c1edb11f 2024-02-10T07:39:47.241 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 07:39:46 smithi070 systemd[1]: Started Ceph node-exporter.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:39:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:46 smithi155 ceph-mon[32688]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:48.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[39923]: Deploying daemon node-exporter.b on smithi155 2024-02-10T07:39:48.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:48 smithi070 ceph-mon[29548]: Deploying daemon node-exporter.b on smithi155 2024-02-10T07:39:48.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:48 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:48 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:48 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:48.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:48 smithi155 ceph-mon[32688]: Deploying daemon node-exporter.b on smithi155 2024-02-10T07:39:48.506 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:48 smithi155 bash[63647]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T07:39:48.850 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:48 smithi155 bash[63647]: Getting image source signatures 2024-02-10T07:39:48.850 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:48 smithi155 bash[63647]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T07:39:48.850 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:48 smithi155 bash[63647]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T07:39:48.850 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:48 smithi155 bash[63647]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T07:39:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:49 smithi155 ceph-mon[32688]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:49.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:39:48 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:39:49.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:49 smithi070 ceph-mon[29548]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:49.391 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:49 smithi070 ceph-mon[39923]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:50.094 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:49 smithi155 bash[63647]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T07:39:50.094 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:49 smithi155 bash[63647]: Writing manifest to image destination 2024-02-10T07:39:50.347 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 podman[63647]: 2024-02-10 07:39:50.248138836 +0000 UTC m=+2.117468894 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T07:39:50.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:50.406 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:50.407 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:50.625 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 podman[63647]: 2024-02-10 07:39:50.348191548 +0000 UTC m=+2.217521652 container create 9284088414a559955b951a8edefde7afd09c536315798d9224f92d9e2ebc566d (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:39:50.625 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 podman[63647]: 2024-02-10 07:39:50.558457521 +0000 UTC m=+2.427787580 container init 9284088414a559955b951a8edefde7afd09c536315798d9224f92d9e2ebc566d (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:39:50.625 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 podman[63647]: 2024-02-10 07:39:50.561245157 +0000 UTC m=+2.430575216 container start 9284088414a559955b951a8edefde7afd09c536315798d9224f92d9e2ebc566d (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:39:50.625 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.565Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.565Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.566Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.566Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.566Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.566Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T07:39:50.626 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T07:39:50.627 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T07:39:50.628 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T07:39:50.629 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T07:39:50.630 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T07:39:50.630 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T07:39:50.630 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b[63696]: ts=2024-02-10T07:39:50.567Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T07:39:50.630 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 bash[63647]: 9284088414a559955b951a8edefde7afd09c536315798d9224f92d9e2ebc566d 2024-02-10T07:39:51.005 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 07:39:50 smithi155 systemd[1]: Started Ceph node-exporter.b for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:39:51.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[39923]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:51.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[29548]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:51 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:39:51.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:51 smithi155 ceph-mon[32688]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:51.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:51 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:51 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:51 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:51 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:51 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:51.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:51 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:39:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:39:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:39:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:39:53.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:53 smithi070 ceph-mon[39923]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:53.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:53.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:53.471 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:53 smithi070 ceph-mon[29548]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:53.471 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:53.471 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:53.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:53 smithi155 ceph-mon[32688]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:53.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:53.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:39:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:54 smithi070 ceph-mon[39923]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:54 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:54 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:54 smithi070 ceph-mon[29548]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:54 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:54 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:55.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:54 smithi155 ceph-mon[32688]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:54 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:54 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:39:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:39:56.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:39:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:39:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:39:56.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:56 smithi070 ceph-mon[29548]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T07:39:56.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:56 smithi070 ceph-mon[29548]: Reconfiguring daemon alertmanager.a on smithi070 2024-02-10T07:39:56.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:56 smithi070 ceph-mon[29548]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:57.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:56 smithi070 ceph-mon[39923]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T07:39:57.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:56 smithi070 ceph-mon[39923]: Reconfiguring daemon alertmanager.a on smithi070 2024-02-10T07:39:57.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:56 smithi070 ceph-mon[39923]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:56 smithi155 ceph-mon[32688]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T07:39:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:56 smithi155 ceph-mon[32688]: Reconfiguring daemon alertmanager.a on smithi070 2024-02-10T07:39:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:56 smithi155 ceph-mon[32688]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:39:59.082 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:58 smithi070 ceph-mon[39923]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:59.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:58 smithi070 ceph-mon[29548]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:59.083 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:58 smithi070 systemd[1]: Stopping Ceph alertmanager.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:39:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:58 smithi155 ceph-mon[32688]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:39:59.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:39:58 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:39:59.349 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[75286]: ts=2024-02-10T07:39:59.082Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-10T07:39:59.349 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 podman[76374]: 2024-02-10 07:39:59.102595224 +0000 UTC m=+0.117648057 container died f8d8f1bec1f5bc792b1d5567c658270662d6acaed980557d6c8a8dda8476e024 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:59.641 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 podman[76374]: 2024-02-10 07:39:59.449044451 +0000 UTC m=+0.464097316 container cleanup f8d8f1bec1f5bc792b1d5567c658270662d6acaed980557d6c8a8dda8476e024 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:59.641 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 bash[76374]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T07:39:59.945 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:39:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:39:59.946 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 podman[76385]: 2024-02-10 07:39:59.641675841 +0000 UTC m=+0.537822258 container remove f8d8f1bec1f5bc792b1d5567c658270662d6acaed980557d6c8a8dda8476e024 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:39:59.946 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 podman[76385]: 2024-02-10 07:39:59.709016494 +0000 UTC m=+0.605162929 volume remove f304339dce8a93b642c92a4c3f438ca75a4c2dff6e55246c40734ed0df454792 2024-02-10T07:40:00.226 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:39:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:00.227 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-10T07:40:00.227 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 systemd[1]: Stopped Ceph alertmanager.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:40:00.227 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:39:59 smithi070 systemd[1]: Starting Ceph alertmanager.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:40:00.227 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 podman[76471]: 2024-02-10 07:40:00.226597164 +0000 UTC m=+0.113969605 volume create 9f937efc2bb820bbe11b45c33ae7ad4272d2ac28ca2aa5e131d5a2fb122efb28 2024-02-10T07:40:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:39:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:00.491 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 podman[76471]: 2024-02-10 07:40:00.134588558 +0000 UTC m=+0.021961002 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-10T07:40:00.491 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 podman[76471]: 2024-02-10 07:40:00.343951954 +0000 UTC m=+0.231324389 container create d1b6c45deea91b7e755d36a4138dd4289e0e678df501703dfdc7fa97c2d92c71 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 podman[76471]: 2024-02-10 07:40:00.671886236 +0000 UTC m=+0.559258689 container init d1b6c45deea91b7e755d36a4138dd4289e0e678df501703dfdc7fa97c2d92c71 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 podman[76471]: 2024-02-10 07:40:00.676581215 +0000 UTC m=+0.563953653 container start d1b6c45deea91b7e755d36a4138dd4289e0e678df501703dfdc7fa97c2d92c71 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.692Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.693Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.694Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.70 port=9094 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.696Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.735Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T07:40:00.947 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.736Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T07:40:00.948 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.740Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-10T07:40:00.948 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:00.740Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-10T07:40:00.948 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 bash[76471]: d1b6c45deea91b7e755d36a4138dd4289e0e678df501703dfdc7fa97c2d92c71 2024-02-10T07:40:00.948 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:00 smithi070 systemd[1]: Started Ceph alertmanager.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:40:00.948 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[29548]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:00.948 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[29548]: overall HEALTH_OK 2024-02-10T07:40:00.948 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:00.948 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[39923]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[39923]: overall HEALTH_OK 2024-02-10T07:40:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:00 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:00 smithi155 ceph-mon[32688]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:00 smithi155 ceph-mon[32688]: overall HEALTH_OK 2024-02-10T07:40:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:00 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:00 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:02.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:01 smithi070 ceph-mon[39923]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T07:40:02.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:01 smithi070 ceph-mon[39923]: Reconfiguring daemon prometheus.a on smithi155 2024-02-10T07:40:02.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:01 smithi070 ceph-mon[29548]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T07:40:02.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:01 smithi070 ceph-mon[29548]: Reconfiguring daemon prometheus.a on smithi155 2024-02-10T07:40:02.242 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:40:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:40:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:01 smithi155 ceph-mon[32688]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T07:40:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:01 smithi155 ceph-mon[32688]: Reconfiguring daemon prometheus.a on smithi155 2024-02-10T07:40:02.698 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:02.697Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000820602s 2024-02-10T07:40:03.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:02 smithi070 ceph-mon[39923]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:03.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:02 smithi070 ceph-mon[29548]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:02 smithi155 ceph-mon[32688]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:04 smithi070 ceph-mon[29548]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:05.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:04 smithi070 ceph-mon[39923]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:05.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:04 smithi155 ceph-mon[32688]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:07.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:06 smithi155 ceph-mon[32688]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:07.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:06 smithi070 ceph-mon[39923]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:06 smithi070 ceph-mon[29548]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:07 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:08.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:07 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:07 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:09.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:08 smithi070 ceph-mon[39923]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:09.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:08 smithi070 ceph-mon[29548]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:09.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:40:08 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:40:09.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:08 smithi155 ceph-mon[32688]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:09.741 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 systemd[1]: Stopping Ceph prometheus.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:40:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T07:40:10.005 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.741Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-10T07:40:10.006 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.742Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-10T07:40:10.006 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.742Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T07:40:10.006 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.742Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-10T07:40:10.006 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[61484]: ts=2024-02-10T07:40:09.742Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-10T07:40:10.006 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:09 smithi155 podman[64611]: 2024-02-10 07:40:09.752028689 +0000 UTC m=+0.134846095 container died 12f568af1e4908067bf08193d255df812b81a9d0e1e7eaef22b08b0011a18134 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:10.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:09 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:09 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:10.383 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 podman[64611]: 2024-02-10 07:40:10.155100445 +0000 UTC m=+0.537917846 container cleanup 12f568af1e4908067bf08193d255df812b81a9d0e1e7eaef22b08b0011a18134 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:10.383 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 bash[64611]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T07:40:10.755 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 podman[64625]: 2024-02-10 07:40:10.383996392 +0000 UTC m=+0.634700606 container remove 12f568af1e4908067bf08193d255df812b81a9d0e1e7eaef22b08b0011a18134 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:10.755 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-02-10T07:40:10.755 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 systemd[1]: Stopped Ceph prometheus.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:40:10.755 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@prometheus.a.service: Consumed 1.085s CPU time. 2024-02-10T07:40:10.755 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 systemd[1]: Starting Ceph prometheus.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T07:40:10.968 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 07:40:10 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T07:40:10.701Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003936093s 2024-02-10T07:40:11.087 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:10 smithi155 ceph-mon[32688]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:11.088 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 podman[64718]: 2024-02-10 07:40:10.734253952 +0000 UTC m=+0.018739211 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-10T07:40:11.088 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:10 smithi155 podman[64718]: 2024-02-10 07:40:10.85171767 +0000 UTC m=+0.136202928 container create 699fc1bc5a90ba308c649c08c7f105c5a5c31ea651df46458f14aed63faff6e7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:11.088 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 podman[64718]: 2024-02-10 07:40:11.061968373 +0000 UTC m=+0.346453632 container init 699fc1bc5a90ba308c649c08c7f105c5a5c31ea651df46458f14aed63faff6e7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:11.088 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 podman[64718]: 2024-02-10 07:40:11.064730939 +0000 UTC m=+0.349216214 container start 699fc1bc5a90ba308c649c08c7f105c5a5c31ea651df46458f14aed63faff6e7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:40:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:10 smithi070 ceph-mon[29548]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:11.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:10 smithi070 ceph-mon[39923]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:11.505 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.088Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.088Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.088Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi155 (none))" 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.088Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.088Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.089Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.090Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.091Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.091Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.093Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.093Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.761µs 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.093Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.094Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-10T07:40:11.506 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.094Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.094Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=20.562µs wal_replay_duration=499.395µs wbl_replay_duration=87ns total_replay_duration=538.238µs 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.095Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.095Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.095Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.118Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=23.839744ms db_storage=820ns remote_storage=1.203µs web_handler=661ns query_engine=585ns scrape=367.893µs scrape_sd=101.113µs notify=10.177µs notify_sd=10.107µs rules=23.0665ms tracing=6.013µs 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.118Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T07:40:11.118Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 bash[64718]: 699fc1bc5a90ba308c649c08c7f105c5a5c31ea651df46458f14aed63faff6e7 2024-02-10T07:40:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 07:40:11 smithi155 systemd[1]: Started Ceph prometheus.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T07:40:11.649 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STOPPING 2024-02-10T07:40:11.649 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STOPPED 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STARTING 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Serving on http://:::9283 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STARTED 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STOPPING 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STOPPED 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STARTING 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Serving on http://:::9283 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STARTED 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STOPPING 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STOPPED 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STARTING 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Serving on http://:::9283 2024-02-10T07:40:11.650 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:11 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:07:40:11] ENGINE Bus STARTED 2024-02-10T07:40:12.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T07:40:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:40:12.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T07:40:12.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:12.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:40:13.686 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:13 smithi155 ceph-mon[32688]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:13 smithi070 ceph-mon[39923]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:13.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:13 smithi070 ceph-mon[29548]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:15 smithi070 ceph-mon[39923]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:15 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:15 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:15 smithi070 ceph-mon[29548]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:15 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:15 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:15 smithi155 ceph-mon[32688]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:15 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:15 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:40:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:40:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:40:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:40:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:16 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.750 INFO:tasks.workunit.client.0.smithi070.stderr:Note: switching to '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'. 2024-02-10T07:40:16.750 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-02-10T07:40:16.750 INFO:tasks.workunit.client.0.smithi070.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-10T07:40:16.750 INFO:tasks.workunit.client.0.smithi070.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-10T07:40:16.750 INFO:tasks.workunit.client.0.smithi070.stderr:state without impacting any branches by switching back to a branch. 2024-02-10T07:40:16.750 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-02-10T07:40:16.750 INFO:tasks.workunit.client.0.smithi070.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr: git switch -c 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr:Or undo this operation with: 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr: git switch - 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr: 2024-02-10T07:40:16.751 INFO:tasks.workunit.client.0.smithi070.stderr:HEAD is now at 290212a1bfd Merge pull request #53521 from xxhdx1985126/wip-62857 2024-02-10T07:40:16.755 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-10T07:40:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:16 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:16 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:16 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:40:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:16 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:40:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:16 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:16.776 INFO:tasks.workunit.client.0.smithi070.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-10T07:40:16.778 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-10T07:40:16.779 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-10T07:40:17.529 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-10T07:40:17.613 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-10T07:40:17.653 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-10T07:40:17.654 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-10T07:40:17.655 INFO:tasks.workunit.client.0.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-10T07:40:17.693 INFO:tasks.workunit.client.0.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-10T07:40:17.696 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-02-10T07:40:17.696 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-10T07:40:17.752 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-10T07:40:17.754 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-10T07:40:17.754 DEBUG:teuthology.orchestra.run.smithi070:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-10T07:40:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:17 smithi155 ceph-mon[32688]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ parallel=1 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' '' = --serial ']' 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ crimson=0 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' '' = --crimson ']' 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ color= 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -t 1 ']' 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-10T07:40:17.821 INFO:tasks.workunit.client.0.smithi070.stderr:+ GTEST_OUTPUT=xml:/home/ubuntu/cephtest/archive/gtest_xml_report 2024-02-10T07:40:17.822 INFO:tasks.workunit.client.0.smithi070.stderr:+ declare -A pids 2024-02-10T07:40:17.822 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.822 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.822 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_aio 2024-02-10T07:40:17.822 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_aio' 2024-02-10T07:40:17.822 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_aio 2024-02-10T07:40:17.822 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.823 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_aio 2024-02-10T07:40:17.823 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77197 2024-02-10T07:40:17.823 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_aio on pid 77197' 2024-02-10T07:40:17.823 INFO:tasks.workunit.client.0.smithi070.stdout:test api_aio on pid 77197 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77197 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_aio_pp 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_aio_pp' 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_aio_pp 2024-02-10T07:40:17.826 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.827 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_aio_pp 2024-02-10T07:40:17.828 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77204 2024-02-10T07:40:17.829 INFO:tasks.workunit.client.0.smithi070.stdout:test api_aio_pp on pid 77204 2024-02-10T07:40:17.832 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_aio_pp on pid 77204' 2024-02-10T07:40:17.832 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77204 2024-02-10T07:40:17.832 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.832 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_io 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_io' 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_io 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.833 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.834 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.835 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_io 2024-02-10T07:40:17.836 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77213 2024-02-10T07:40:17.837 INFO:tasks.workunit.client.0.smithi070.stdout:test api_io on pid 77213 2024-02-10T07:40:17.841 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_io on pid 77213' 2024-02-10T07:40:17.841 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77213 2024-02-10T07:40:17.841 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.841 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.841 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-10T07:40:17.841 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_io_pp 2024-02-10T07:40:17.841 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_io_pp' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_io_pp 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.842 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.843 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.844 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_io_pp 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77225 2024-02-10T07:40:17.845 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_io_pp on pid 77225' 2024-02-10T07:40:17.846 INFO:tasks.workunit.client.0.smithi070.stdout:test api_io_pp on pid 77225 2024-02-10T07:40:17.851 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77225 2024-02-10T07:40:17.851 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.851 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.851 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-10T07:40:17.851 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_asio 2024-02-10T07:40:17.851 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_asio' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_asio 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.852 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.853 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.854 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.855 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.856 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_asio 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77238 2024-02-10T07:40:17.857 INFO:tasks.workunit.client.0.smithi070.stdout:test api_asio on pid 77238 2024-02-10T07:40:17.865 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_asio on pid 77238' 2024-02-10T07:40:17.865 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77238 2024-02-10T07:40:17.865 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.865 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_list 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_list' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_list 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.866 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.867 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.868 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.869 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.870 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.871 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.872 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_list 2024-02-10T07:40:17.873 INFO:tasks.workunit.client.0.smithi070.stdout:test api_list on pid 77255 2024-02-10T07:40:17.878 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77255 2024-02-10T07:40:17.878 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77197/exe 2024-02-10T07:40:17.879 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_list on pid 77255' 2024-02-10T07:40:17.879 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77255 2024-02-10T07:40:17.879 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.879 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.879 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-10T07:40:17.879 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.879 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_lock 2024-02-10T07:40:17.880 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_lock' 2024-02-10T07:40:17.880 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.880 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_lock 2024-02-10T07:40:17.880 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.880 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.880 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.880 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.881 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.882 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77204/exe 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_aio: /' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.887 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.889 INFO:tasks.workunit.client.0.smithi070.stdout:test api_lock on pid 77275 2024-02-10T07:40:17.891 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_lock 2024-02-10T07:40:17.891 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77275 2024-02-10T07:40:17.891 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_lock on pid 77275' 2024-02-10T07:40:17.891 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77275 2024-02-10T07:40:17.891 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.891 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.891 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_lock_pp 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_lock_pp' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.892 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.893 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.894 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.895 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.896 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.897 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_lock_pp 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.898 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.899 INFO:tasks.workunit.client.0.smithi070.stdout:test api_lock_pp on pid 77294 2024-02-10T07:40:17.900 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.900 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.900 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.900 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.900 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.900 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.900 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_lock_pp 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.901 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77294 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_lock_pp on pid 77294' 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77294 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_misc 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_misc' 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77213/exe 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.902 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_misc 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.903 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.904 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.905 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.906 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.907 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77225/exe 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.908 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_io.log 2024-02-10T07:40:17.909 INFO:tasks.workunit.client.0.smithi070.stdout:test api_misc on pid 77317 2024-02-10T07:40:17.915 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_io: /' 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.916 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_misc 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77317 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_misc on pid 77317' 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77317 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.917 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.918 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.919 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.920 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.921 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-10T07:40:17.922 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_misc_pp 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_misc_pp' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.923 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.924 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77238/exe 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.925 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.926 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.927 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77255/exe 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.928 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.929 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_misc_pp 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.930 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_asio: /' 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.931 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.932 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.933 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_list.log 2024-02-10T07:40:17.934 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_list: /' 2024-02-10T07:40:17.934 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.934 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.934 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_misc_pp 2024-02-10T07:40:17.934 INFO:tasks.workunit.client.0.smithi070.stdout:test api_misc_pp on pid 77354 2024-02-10T07:40:17.940 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77354 2024-02-10T07:40:17.940 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_misc_pp on pid 77354' 2024-02-10T07:40:17.940 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77354 2024-02-10T07:40:17.940 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.940 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.941 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.942 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.943 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77275/exe 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.944 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.945 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.946 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.947 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.948 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_tier_pp 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_tier_pp' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_lock: /' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.949 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.950 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.951 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77294/exe 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.952 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_tier_pp 2024-02-10T07:40:17.953 INFO:tasks.workunit.client.0.smithi070.stdout:test api_tier_pp on pid 77411 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_tier_pp 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77411 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_tier_pp on pid 77411' 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77411 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-10T07:40:17.956 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.957 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.958 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_pool 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_pool' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.959 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.960 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.961 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.962 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.963 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_pool 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.964 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_pool 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77467 2024-02-10T07:40:17.965 INFO:tasks.workunit.client.0.smithi070.stdout:test api_pool on pid 77467 2024-02-10T07:40:17.970 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_pool on pid 77467' 2024-02-10T07:40:17.970 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77467 2024-02-10T07:40:17.970 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.970 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.970 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_snapshots 2024-02-10T07:40:17.970 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_snapshots' 2024-02-10T07:40:17.970 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77354/exe 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-10T07:40:17.971 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77317/exe 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.972 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.973 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:17.974 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_snapshots 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.975 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_misc: /' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.976 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.977 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.977 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.977 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.977 INFO:tasks.workunit.client.0.smithi070.stdout:test api_snapshots on pid 77514 2024-02-10T07:40:17.979 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.979 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.979 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.979 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_snapshots 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77514 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_snapshots on pid 77514' 2024-02-10T07:40:17.980 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77514 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_snapshots_pp 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_snapshots_pp' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.981 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.982 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.983 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.984 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.984 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_snapshots_pp 2024-02-10T07:40:17.984 INFO:tasks.workunit.client.0.smithi070.stdout:test api_snapshots_pp on pid 77536 2024-02-10T07:40:17.988 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:17.988 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:17.988 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:17.988 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_snapshots_pp 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:17.989 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77536 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_snapshots_pp on pid 77536' 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77536 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_stat 2024-02-10T07:40:17.990 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_stat' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.991 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:17.992 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.993 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.994 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.995 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.996 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77467/exe 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_stat 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_stat 2024-02-10T07:40:17.997 INFO:tasks.workunit.client.0.smithi070.stdout:test api_stat on pid 77560 2024-02-10T07:40:18.006 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.006 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.006 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.006 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.006 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.006 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.006 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77560 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_stat on pid 77560' 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77560 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.007 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77411/exe 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.008 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.009 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.010 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_stat_pp 2024-02-10T07:40:18.011 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_stat_pp' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.012 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.013 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.014 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.015 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_pool: /' 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.016 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.017 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.018 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.019 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.020 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77514/exe 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.021 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.022 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.023 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2024-02-10T07:40:18.024 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.025 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77560/exe 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77536/exe 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.026 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-10T07:40:18.027 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_stat: /' 2024-02-10T07:40:18.028 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.028 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2024-02-10T07:40:18.028 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_stat_pp 2024-02-10T07:40:18.028 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-10T07:40:18.028 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-10T07:40:18.028 INFO:tasks.workunit.client.0.smithi070.stdout:test api_stat_pp on pid 77627 2024-02-10T07:40:18.030 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_stat_pp 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77627 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_stat_pp on pid 77627' 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77627 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_watch_notify 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_watch_notify' 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.031 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.032 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_watch_notify 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.033 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.034 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.035 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.035 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.035 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.035 INFO:tasks.workunit.client.0.smithi070.stdout:test api_watch_notify on pid 77660 2024-02-10T07:40:18.038 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_watch_notify 2024-02-10T07:40:18.038 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77660 2024-02-10T07:40:18.038 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_watch_notify on pid 77660' 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77660 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_watch_notify_pp 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_watch_notify_pp' 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.039 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.040 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_watch_notify_pp 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.041 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.042 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.043 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.044 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.045 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.045 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.045 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.045 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.045 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_watch_notify_pp 2024-02-10T07:40:18.045 INFO:tasks.workunit.client.0.smithi070.stdout:test api_watch_notify_pp on pid 77714 2024-02-10T07:40:18.046 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77714 2024-02-10T07:40:18.046 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_watch_notify_pp on pid 77714' 2024-02-10T07:40:18.046 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77714 2024-02-10T07:40:18.046 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.046 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.046 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-10T07:40:18.046 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_cmd 2024-02-10T07:40:18.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:17 smithi070 ceph-mon[29548]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:18.047 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:17 smithi070 ceph-mon[39923]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_cmd' 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_cmd 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.049 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.050 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.051 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.052 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77627/exe 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.053 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.054 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_cmd 2024-02-10T07:40:18.054 INFO:tasks.workunit.client.0.smithi070.stdout:test api_cmd on pid 77745 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77745 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_cmd on pid 77745' 2024-02-10T07:40:18.058 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77745 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_cmd_pp 2024-02-10T07:40:18.059 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_cmd_pp' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.060 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.061 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_cmd_pp 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.062 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.063 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.064 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77660/exe 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.066 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.067 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.068 INFO:tasks.workunit.client.0.smithi070.stdout:test api_cmd_pp on pid 77794 2024-02-10T07:40:18.070 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_cmd_pp 2024-02-10T07:40:18.070 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77794 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_cmd_pp on pid 77794' 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77794 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.071 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_service 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_service' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.072 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.073 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_service 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.074 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.075 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.076 INFO:tasks.workunit.client.0.smithi070.stdout:test api_service on pid 77817 2024-02-10T07:40:18.078 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_service 2024-02-10T07:40:18.078 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77817 2024-02-10T07:40:18.078 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_service on pid 77817' 2024-02-10T07:40:18.078 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77817 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_service_pp 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_service_pp' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_service_pp 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.079 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.080 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.081 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.082 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.083 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.084 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77714/exe 2024-02-10T07:40:18.085 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.086 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.086 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.086 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.086 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.086 INFO:tasks.workunit.client.0.smithi070.stdout:test api_service_pp on pid 77832 2024-02-10T07:40:18.088 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.088 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_service_pp 2024-02-10T07:40:18.089 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.089 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77832 2024-02-10T07:40:18.089 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_service_pp on pid 77832' 2024-02-10T07:40:18.089 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77832 2024-02-10T07:40:18.089 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.089 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.089 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_c_write_operations 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_c_write_operations' 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.090 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.091 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.092 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.093 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_c_write_operations 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77745/exe 2024-02-10T07:40:18.094 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2024-02-10T07:40:18.095 INFO:tasks.workunit.client.0.smithi070.stdout:test api_c_write_operations on pid 77846 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_c_write_operations 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77846 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_c_write_operations on pid 77846' 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77846 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.102 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_c_read_operations 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_c_read_operations' 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.103 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.104 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.105 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_cmd: /' 2024-02-10T07:40:18.106 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_c_read_operations 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.107 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.108 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.109 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.110 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.111 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.112 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.113 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77794/exe 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.114 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.115 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-10T07:40:18.116 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.116 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.116 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.116 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.116 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.116 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.117 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.118 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.119 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.120 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.121 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77817/exe 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.122 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.123 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.124 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.124 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.124 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2024-02-10T07:40:18.124 INFO:tasks.workunit.client.0.smithi070.stdout:test api_c_read_operations on pid 77893 2024-02-10T07:40:18.128 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_c_read_operations 2024-02-10T07:40:18.128 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.128 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.128 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.128 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.128 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77893 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_c_read_operations on pid 77893' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77893 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.129 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_service: /' 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77846/exe 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.130 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s api_cls_remote_reads 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' api_cls_remote_reads' 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo api_cls_remote_reads 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.131 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_service.log 2024-02-10T07:40:18.132 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.133 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-10T07:40:18.134 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.134 INFO:tasks.workunit.client.0.smithi070.stdout:test api_cls_remote_reads on pid 77925 2024-02-10T07:40:18.138 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=api_cls_remote_reads 2024-02-10T07:40:18.138 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77925 2024-02-10T07:40:18.138 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test api_cls_remote_reads on pid 77925' 2024-02-10T07:40:18.138 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77925 2024-02-10T07:40:18.138 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.138 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.139 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.140 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.141 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.142 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.143 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77832/exe 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.144 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s list_parallel 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' list_parallel' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.145 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.146 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.147 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.148 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo list_parallel 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.149 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.150 INFO:tasks.workunit.client.0.smithi070.stdout:test list_parallel on pid 77978 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=list_parallel 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77978 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test list_parallel on pid 77978' 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=77978 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.154 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.155 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s open_pools_parallel 2024-02-10T07:40:18.156 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' open_pools_parallel' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo open_pools_parallel 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.157 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.158 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.159 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.160 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.161 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.161 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.161 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.161 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.161 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.161 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.161 INFO:tasks.workunit.client.0.smithi070.stdout:test open_pools_parallel on pid 78013 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=open_pools_parallel 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78013 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test open_pools_parallel on pid 78013' 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78013 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77925/exe 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.164 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.165 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.166 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s delete_pools_parallel 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.167 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' delete_pools_parallel' 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo delete_pools_parallel 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77893/exe 2024-02-10T07:40:18.168 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.169 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.170 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.170 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.170 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=delete_pools_parallel 2024-02-10T07:40:18.170 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.170 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.170 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.170 INFO:tasks.workunit.client.0.smithi070.stdout:test delete_pools_parallel on pid 78036 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78036 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test delete_pools_parallel on pid 78036' 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78036 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.175 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s cls 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' cls' 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo cls 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-10T07:40:18.176 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.177 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.178 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.178 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.178 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.178 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.178 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.178 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.178 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.179 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.180 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.181 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.182 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.183 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/77978/exe 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.184 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.185 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.186 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.187 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.187 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.187 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.187 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.187 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2024-02-10T07:40:18.187 INFO:tasks.workunit.client.0.smithi070.stdout:test cls on pid 78070 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=cls 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78070 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test cls on pid 78070' 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78070 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.192 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.193 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s cmd 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' cmd' 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.194 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.195 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ list_parallel: /' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.196 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.197 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.198 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo cmd 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.199 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.200 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.201 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=cmd 2024-02-10T07:40:18.202 INFO:tasks.workunit.client.0.smithi070.stdout:test cmd on pid 78124 2024-02-10T07:40:18.203 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78124 2024-02-10T07:40:18.203 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test cmd on pid 78124' 2024-02-10T07:40:18.203 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.203 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78124 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.204 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.205 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s handler_error 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' handler_error' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.206 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo handler_error 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.207 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78036/exe 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.208 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.209 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.210 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-10T07:40:18.211 INFO:tasks.workunit.client.0.smithi070.stdout:test handler_error on pid 78160 2024-02-10T07:40:18.213 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=handler_error 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78160 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test handler_error on pid 78160' 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78160 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78070/exe 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.214 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.215 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.215 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.215 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.215 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.215 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.215 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s io 2024-02-10T07:40:18.216 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.216 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.216 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.216 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.216 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.216 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.217 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.217 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.217 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.217 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.217 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.217 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.218 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.218 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.218 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.218 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.218 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.218 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.218 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_cls.log 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' io' 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78013/exe 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.219 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.220 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2024-02-10T07:40:18.220 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ cls: /' 2024-02-10T07:40:18.220 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_cls 2024-02-10T07:40:18.220 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.220 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo io 2024-02-10T07:40:18.220 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.220 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.221 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.221 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.221 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.221 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.221 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.221 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.221 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.222 INFO:tasks.workunit.client.0.smithi070.stdout:test io on pid 78208 2024-02-10T07:40:18.226 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=io 2024-02-10T07:40:18.226 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78208 2024-02-10T07:40:18.226 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test io on pid 78208' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78208 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.227 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.228 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s ec_io 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' ec_io' 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.229 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.230 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo ec_io 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.231 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.232 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.233 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.233 INFO:tasks.workunit.client.0.smithi070.stdout:test ec_io on pid 78221 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=ec_io 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78221 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test ec_io on pid 78221' 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78221 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.239 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.240 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.241 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s list 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' list' 2024-02-10T07:40:18.242 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.243 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.244 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.245 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo list 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.246 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.247 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.248 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.249 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.250 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.251 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.252 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78208/exe 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.253 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.254 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.254 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.254 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.254 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.254 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.254 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.254 INFO:tasks.workunit.client.0.smithi070.stdout:test list on pid 78308 2024-02-10T07:40:18.257 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=list 2024-02-10T07:40:18.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78308 2024-02-10T07:40:18.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test list on pid 78308' 2024-02-10T07:40:18.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78308 2024-02-10T07:40:18.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.257 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.258 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s ec_list 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' ec_list' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_io 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.259 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ io: /' 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_io.log 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.260 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.261 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78124/exe 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.262 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.263 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.264 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo ec_list 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_cmd.log 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ cmd: /' 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_cmd 2024-02-10T07:40:18.265 INFO:tasks.workunit.client.0.smithi070.stdout:test ec_list on pid 78351 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=ec_list 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78351 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test ec_list on pid 78351' 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78351 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.268 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.269 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s misc 2024-02-10T07:40:18.270 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' misc' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.271 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.272 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78160/exe 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.273 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.274 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.275 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.275 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_handler_error.log 2024-02-10T07:40:18.275 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_handler_error 2024-02-10T07:40:18.275 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo misc 2024-02-10T07:40:18.275 INFO:tasks.workunit.client.0.smithi070.stdout:test misc on pid 78372 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=misc 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78372 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test misc on pid 78372' 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78372 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-02-10T07:40:18.277 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s pool 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' pool' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo pool 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ handler_error: /' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.278 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.279 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.280 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.281 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.282 INFO:tasks.workunit.client.0.smithi070.stdout:test pool on pid 78407 2024-02-10T07:40:18.284 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=pool 2024-02-10T07:40:18.284 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78407 2024-02-10T07:40:18.284 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test pool on pid 78407' 2024-02-10T07:40:18.284 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78407 2024-02-10T07:40:18.284 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.284 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.284 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s read_operations 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.285 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.286 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' read_operations' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.287 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.288 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.289 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo read_operations 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.290 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.291 INFO:tasks.workunit.client.0.smithi070.stdout:test read_operations on pid 78444 2024-02-10T07:40:18.293 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=read_operations 2024-02-10T07:40:18.293 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78444 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test read_operations on pid 78444' 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78444 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.294 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s snapshots 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' snapshots' 2024-02-10T07:40:18.295 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.296 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.297 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.298 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo snapshots 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78221/exe 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.299 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.300 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.301 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-02-10T07:40:18.302 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ ec_io: /' 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_ec_io.log 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.303 INFO:tasks.workunit.client.0.smithi070.stdout:test snapshots on pid 78475 2024-02-10T07:40:18.308 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.308 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.308 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=snapshots 2024-02-10T07:40:18.308 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_ec_io 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78475 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test snapshots on pid 78475' 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78475 2024-02-10T07:40:18.309 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s watch_notify 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' watch_notify' 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.310 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.311 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.312 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.313 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78351/exe 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.314 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.315 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78372/exe 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo watch_notify 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78308/exe 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.316 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.317 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.318 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_misc.log 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ misc: /' 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_ec_list 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_ec_list.log 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.319 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.320 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.320 INFO:tasks.workunit.client.0.smithi070.stdout:test watch_notify on pid 78513 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=watch_notify 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78513 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test watch_notify on pid 78513' 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_list 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78513 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ ec_list: /' 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_misc 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.323 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.324 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ list: /' 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:++ printf %25s write_operations 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_list.log 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:+ r=' write_operations' 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.325 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.326 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.327 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.328 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.329 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++ echo write_operations 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.330 INFO:tasks.workunit.client.0.smithi070.stderr:++ awk '{print $1}' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.331 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.332 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.333 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.333 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.333 INFO:tasks.workunit.client.0.smithi070.stdout:test write_operations on pid 78567 2024-02-10T07:40:18.342 INFO:tasks.workunit.client.0.smithi070.stderr:+ ff=write_operations 2024-02-10T07:40:18.342 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.342 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78567 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo 'test write_operations on pid 78567' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:+ pids[$f]=78567 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.343 INFO:tasks.workunit.client.0.smithi070.stderr:+ ret=0 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77411 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77411 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.344 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78407/exe 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -f /etc/bashrc ']' 2024-02-10T07:40:18.345 INFO:tasks.workunit.client.0.smithi070.stderr:+ . /etc/bashrc 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -z '' ']' 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:++ BASHRCSOURCED=Y 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:++ shopt -q login_shell 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.346 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:+++ umask 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' 0022 -eq 0 ']' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ SHELL=/bin/bash 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-10T07:40:18.347 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.348 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_pool.log 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ pool: /' 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_pool 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.349 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'grep=grep --color=auto' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.350 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorls.sh 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' '!' -t 0 ']' 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:+++ return 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export S_COLORS=auto 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:+++ S_COLORS=auto 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.351 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.352 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.353 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.354 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-10T07:40:18.355 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.356 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.357 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.358 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.359 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78513/exe 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78444/exe 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78475/exe 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.360 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.361 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.362 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ watch_notify: /' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_watch_notify 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_snapshots.log 2024-02-10T07:40:18.363 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/gawk.sh 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/lang.sh 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ snapshots: /' 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_read_operations 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_read_operations.log 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_snapshots 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/locale 2024-02-10T07:40:18.364 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ read_operations: /' 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:++++ export LANG=en_US.UTF-8 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:++++ LANG=en_US.UTF-8 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ LANG=en_US.UTF-8 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ unset LANG_backup config 2024-02-10T07:40:18.365 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n '' ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' dumb = linux ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/less.sh 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ for i in /etc/profile.d/*.sh 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ '[' '' ']' 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:++ . /etc/profile.d/which2.sh 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-10T07:40:18.366 INFO:tasks.workunit.client.0.smithi070.stderr:+++++ readlink /proc/78567/exe 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:++++ basename /usr/bin/bash 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_declare='declare -f' 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+++ which_opt=-f 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export which_declare 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+++ export -f which 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset i 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:++ unset -f pathmunge 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+ export PATH 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+ unset rc 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph_test_neorados_write_operations 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+ tee ceph_test_neorados_write_operations.log 2024-02-10T07:40:18.367 INFO:tasks.workunit.client.0.smithi070.stderr:+ sed 's/^/ write_operations: /' 2024-02-10T07:40:18.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:18.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2996061503' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T07:40:18.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:18.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2996061503' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T07:40:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:19.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:40:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:40:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2996061503' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T07:40:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:22.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:21 smithi070 ceph-mon[39923]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:22.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:21 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:22.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:21 smithi070 ceph-mon[29548]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:22.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:21 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:21 smithi155 ceph-mon[32688]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:40:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:21 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:40:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:40:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1269203837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/833420104' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[39923]: pgmap v55: 228 pgs: 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:40:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[39923]: osdmap e74: 8 total, 8 up, 8 in 2024-02-10T07:40:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786944343' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi070-77269-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1269203837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/833420104' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[29548]: pgmap v55: 228 pgs: 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[29548]: osdmap e74: 8 total, 8 up, 8 in 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786944343' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi070-77269-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1269203837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/833420104' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:24 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:24 smithi155 ceph-mon[32688]: pgmap v55: 228 pgs: 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:40:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:24 smithi155 ceph-mon[32688]: osdmap e74: 8 total, 8 up, 8 in 2024-02-10T07:40:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786944343' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi070-77269-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: pgmap v56: 228 pgs: 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2137306562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/95310955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2928865431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1700463809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2573128969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480353457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/455779000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2743508943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3226627637' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2615808085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3965790300' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: osdmap e75: 8 total, 8 up, 8 in 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: pgmap v56: 228 pgs: 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2137306562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/95310955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2928865431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1700463809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2573128969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480353457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/455779000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2743508943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3226627637' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2615808085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3965790300' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: osdmap e75: 8 total, 8 up, 8 in 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:26.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:26.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:26.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:26.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:26 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: pgmap v56: 228 pgs: 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:40:27.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2137306562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/95310955' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2928865431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1700463809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2573128969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480353457' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/455779000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2743508943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3226627637' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2615808085' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3965790300' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:27.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: osdmap e75: 8 total, 8 up, 8 in 2024-02-10T07:40:27.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:27.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:27.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:27.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:27.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:27.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:26 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [----------] Global test environment set-up. 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5066 ms) 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ OK ] LibRadosService.RegisterLate (28 ms) 2024-02-10T07:40:27.340 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: cluster: 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: id: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: health: HEALTH_OK 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: services: 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: mgr: y(active, since 105s), standbys: x 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: data: 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-10T07:40:27.341 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: pgs: 132 active+clean 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: io: 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: cluster: 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: id: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: health: HEALTH_WARN 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 3 pool(s) do not have an application enabled 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: services: 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: mgr: y(active, since 107s), standbys: x 2024-02-10T07:40:27.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: data: 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: pools: 9 pools, 228 pgs 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: pgs: 42.105% pgs not active 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 132 active+clean 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 96 creating+peering 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: io: 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: cluster: 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: id: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: health: HEALTH_WARN 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 16 pool(s) do not have an application enabled 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: services: 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: mgr: y(active, since 109s), standbys: x 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:27.344 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: data: 2024-02-10T07:40:27.345 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: pools: 22 pools, 644 pgs 2024-02-10T07:40:27.345 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-10T07:40:27.345 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:27.345 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: pgs: 64.596% pgs unknown 2024-02-10T07:40:27.345 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 14.907% pgs not active 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [----------] Global test environment set-up. 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (4 ms) 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-10T07:40:28.030 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (20 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (15 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (8 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (2 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (2 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (23 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (7 ms) 2024-02-10T07:40:28.031 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (10 ms) 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3 ms) 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [----------] 12 tests from AsioRados (95 ms total) 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [----------] Global test environment tear-down 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (10126 ms total) 2024-02-10T07:40:28.032 INFO:tasks.workunit.client.0.smithi070.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-10T07:40:28.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-10T07:40:28.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] Global test environment set-up. 2024-02-10T07:40:28.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-10T07:40:28.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-10T07:40:28.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosList.ListObjects (429 ms) 2024-02-10T07:40:28.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-10T07:40:28.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (40 ms) 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset foo1,foo2,foo3 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo1 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo2 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo3 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset foo1,foo4,foo5 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo4 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo5 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo1 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset foo6,foo7 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo7 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo6 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-10T07:40:28.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns1:foo4 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns1:foo5 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns2:foo7 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns2:foo6 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns1:foo1 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: :foo1 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: :foo2 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: :foo3 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (88 ms) 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 1 0 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 10 0 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 13 0 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 7 0 2024-02-10T07:40:28.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 14 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 0 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 15 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 11 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 5 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 8 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 6 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 3 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 4 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 12 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 9 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 2 0 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (66 ms) 2024-02-10T07:40:28.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: x cursor=MIN 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=1 cursor=16:02547ec2:::1:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=10 cursor=16:52ea6a34:::10:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=13 cursor=16:566253c9:::13:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=7 cursor=16:5c6b0b28:::7:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=14 cursor=16:62a1935d:::14:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=0 cursor=16:6cac518f:::0:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=15 cursor=16:863748b0:::15:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=11 cursor=16:89d3ae78:::11:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=5 cursor=16:b29083e3:::5:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=8 cursor=16:bd63b0f1:::8:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=6 cursor=16:c4fdafeb:::6:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=3 cursor=16:cfc208b3:::3:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=4 cursor=16:d83876eb:::4:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=12 cursor=16:de5d7c5f:::12:head 2024-02-10T07:40:28.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=9 cursor=16:e960b815:::9:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > oid=2 cursor=16:f905c69b:::2:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=1 cursor=16:02547ec2:::1:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:02547ec2:::1:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:02547ec2:::1:head -> 1 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=10 cursor=16:52ea6a34:::10:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:52ea6a34:::10:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:52ea6a34:::10:head -> 10 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=13 cursor=16:566253c9:::13:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:566253c9:::13:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:566253c9:::13:head -> 13 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=7 cursor=16:5c6b0b28:::7:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:5c6b0b28:::7:head 2024-02-10T07:40:28.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:5c6b0b28:::7:head -> 7 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=14 cursor=16:62a1935d:::14:head 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:62a1935d:::14:head 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:62a1935d:::14:head -> 14 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=0 cursor=16:6cac518f:::0:head 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:6cac518f:::0:head 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:6cac518f:::0:head -> 0 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=15 cursor=16:863748b0:::15:head 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:863748b0:::15:head 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:863748b0:::15:head -> 15 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=11 cursor=16:89d3ae78:::11:head 2024-02-10T07:40:28.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:89d3ae78:::11:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:89d3ae78:::11:head -> 11 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=5 cursor=16:b29083e3:::5:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:b29083e3:::5:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:b29083e3:::5:head -> 5 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=8 cursor=16:bd63b0f1:::8:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:bd63b0f1:::8:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:bd63b0f1:::8:head -> 8 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=6 cursor=16:c4fdafeb:::6:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:c4fdafeb:::6:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:c4fdafeb:::6:head -> 6 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=3 cursor=16:cfc208b3:::3:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:cfc208b3:::3:head 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:cfc208b3:::3:head -> 3 2024-02-10T07:40:28.459 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=4 cursor=16:d83876eb:::4:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:d83876eb:::4:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:d83876eb:::4:head -> 4 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=12 cursor=16:de5d7c5f:::12:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:de5d7c5f:::12:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:de5d7c5f:::12:head -> 12 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=9 cursor=16:e960b815:::9:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:e960b815:::9:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:e960b815:::9:head -> 9 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : oid=2 cursor=16:f905c69b:::2:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:f905c69b:::2:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:f905c69b:::2:head -> 2 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:de5d7c5f:::12:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:de5d7c5f:::12:head expected=16:de5d7c5f:::12:head 2024-02-10T07:40:28.460 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:de5d7c5f:::12:head -> 12 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=12 expected=12 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:d83876eb:::4:head 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:d83876eb:::4:head expected=16:d83876eb:::4:head 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:d83876eb:::4:head -> 4 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=4 expected=4 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:b29083e3:::5:head 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:b29083e3:::5:head expected=16:b29083e3:::5:head 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:b29083e3:::5:head -> 5 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=5 expected=5 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:f905c69b:::2:head 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:f905c69b:::2:head expected=16:f905c69b:::2:head 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:f905c69b:::2:head -> 2 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=2 expected=2 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:e960b815:::9:head 2024-02-10T07:40:28.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:e960b815:::9:head expected=16:e960b815:::9:head 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:e960b815:::9:head -> 9 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=9 expected=9 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:89d3ae78:::11:head 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:89d3ae78:::11:head expected=16:89d3ae78:::11:head 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:89d3ae78:::11:head -> 11 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=11 expected=11 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:566253c9:::13:head 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:566253c9:::13:head expected=16:566253c9:::13:head 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:566253c9:::13:head -> 13 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=13 expected=13 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:cfc208b3:::3:head 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:cfc208b3:::3:head expected=16:cfc208b3:::3:head 2024-02-10T07:40:28.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:cfc208b3:::3:head -> 3 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : e list_parallel: process_1_[78100]: starting. 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_1_[78100]: creating pool ceph_test_rados_list_parallel.smithi070-78064 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_1_[78100]: created object 0... 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_1_[78100]: created object 25... 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_1_[78100]: created object 49... 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_1_[78100]: finishing. 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_1_[78100]: shutting down. 2024-02-10T07:40:28.607 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_2_[78104]: starting. 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_2_[78104]: listing objects. 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_2_[78104]: listed object 0... 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_2_[78104]: listed object 25... 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_2_[78104]: saw 50 objects 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_2_[78104]: shutting down. 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_3_[79291]: starting. 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_3_[79291]: creating pool ceph_test_rados_list_parallel.smithi070-78064 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_3_[79291]: created object 0... 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_3_[79291]: created object 25... 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_3_[79291]: created object 49... 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_3_[79291]: finishing. 2024-02-10T07:40:28.608 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_3_[79291]: shutting down. 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_4_[79292]: starting. 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_4_[79292]: listing objects. 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_4_[79292]: listed object 0... 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_4_[79292]: listed object 25... 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_4_[79292]: saw 42 objects 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_4_[79292]: shutting down. 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_5_[79293]: starting. 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_5_[79293]: removed 25 objects... 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_5_[79293]: removed half of the objects 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_5_[79293]: removed 50 objects... 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_5_[79293]: removed 50 objects 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_5_[79293]: shutting down. 2024-02-10T07:40:28.609 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_6_[79384]: starting. 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_6_[79384]: creating pool ceph_test_rados_list_parallel.smithi070-78064 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_6_[79384]: created object 0... 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_6_[79384]: created object 25... 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_6_[79384]: created object 49... 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_6_[79384]: finishing. 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_6_[79384]: shutting down. 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_7_[79385]: starting. 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_7_[79385]: listing objects. 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_7_[79385]: listed object 0... 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_7_[79385]: listed object 25... 2024-02-10T07:40:28.610 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_7_[79385]: listed object 50... 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_7_[79385]: saw 58 objects 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_7_[79385]: shutting down. 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_8_[79386]: starting. 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_8_[79386]: added 25 objects... 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_8_[79386]: added half of the objects 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_8_[79386]: added 50 objects... 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_8_[79386]: added 50 objects 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_8_[79386]: shutting down. 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.611 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: pgmap v58: 644 pgs: 416 unknown, 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786944343' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi070-77269-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1488520201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3481796856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2622841185' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/486696585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/841499186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3888870066' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: pgmap v60: 996 pgs: 352 unknown, 512 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:40:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: osdmap e76: 8 total, 8 up, 8 in 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24670 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:28.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: osdmap e77: 8 total, 8 up, 8 in 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34823 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24808 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:28.746 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:28.746 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1097772739' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: pgmap v58: 644 pgs: 416 unknown, 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:40:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786944343' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi070-77269-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1488520201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3481796856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2622841185' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/486696585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/841499186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3888870066' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: pgmap v60: 996 pgs: 352 unknown, 512 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: osdmap e76: 8 total, 8 up, 8 in 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24670 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: osdmap e77: 8 total, 8 up, 8 in 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:28.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34823 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24808 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:28.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1097772739' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: pgmap v58: 644 pgs: 416 unknown, 96 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:40:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786944343' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi070-77269-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1488520201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3481796856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2622841185' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/486696585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/841499186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3888870066' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: pgmap v60: 996 pgs: 352 unknown, 512 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: osdmap e76: 8 total, 8 up, 8 in 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24670 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi070-78508-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi070-78461-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi070-77318-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: osdmap e77: 8 total, 8 up, 8 in 2024-02-10T07:40:28.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34823 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24808 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:28.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1097772739' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T07:40:28.920 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.920 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_9_[79432]: starting. 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_9_[79432]: creating pool ceph_test_rados_list_parallel.smithi070-78064 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_9_[79432]: created object 0... 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_9_[79432]: created object 25... 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_9_[79432]: created object 49... 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_9_[79432]: finishing. 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_9_[79432]: shutting down. 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.921 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.922 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.922 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: starting. 2024-02-10T07:40:28.922 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: listing objects. 2024-02-10T07:40:28.922 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: listed object 0... 2024-02-10T07:40:28.922 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: listed object 25... 2024-02-10T07:40:28.922 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: listed object 50... 2024-02-10T07:40:28.922 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: listed object 75... 2024-02-10T07:40:28.923 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: saw 98 objects 2024-02-10T07:40:28.923 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_10_[79433]: shutting down. 2024-02-10T07:40:28.923 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.923 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.923 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.923 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_13_[79436]: starting. 2024-02-10T07:40:28.923 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_13_[79436]: removed 25 objects... 2024-02-10T07:40:28.924 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_13_[79436]: removed half of the objects 2024-02-10T07:40:28.924 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_13_[79436]: removed 50 objects... 2024-02-10T07:40:28.924 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_13_[79436]: removed 50 objects 2024-02-10T07:40:28.924 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_13_[79436]: shutting down. 2024-02-10T07:40:28.924 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.924 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.924 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.925 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_11_[79434]: starting. 2024-02-10T07:40:28.925 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_11_[79434]: added 25 objects... 2024-02-10T07:40:28.925 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_11_[79434]: added half of the objects 2024-02-10T07:40:28.925 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_11_[79434]: added 50 objects... 2024-02-10T07:40:28.925 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_11_[79434]: added 50 objects 2024-02-10T07:40:28.925 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_11_[79434]: shutting down. 2024-02-10T07:40:28.925 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_12_[79435]: starting. 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_12_[79435]: added 25 objects... 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_12_[79435]: added half of the objects 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_12_[79435]: added 50 objects... 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_12_[79435]: added 50 objects 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_12_[79435]: shutting down. 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.926 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_14_[79529]: starting. 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_14_[79529]: creating pool ceph_test_rados_list_parallel.smithi070-78064 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_14_[79529]: created object 0... 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_14_[79529]: created object 25... 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_14_[79529]: created object 49... 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_14_[79529]: finishing. 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_14_[79529]: shutting down. 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:28.927 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: starting. 2024-02-10T07:40:28.928 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: listing objects. 2024-02-10T07:40:28.928 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: listed object 0... 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: listed object 2 handler_error: Running main() from gmock_main.cc 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [----------] Global test environment set-up. 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (10750 ms) 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [----------] 1 test from neocls_handler_error (10750 ms total) 2024-02-10T07:40:28.974 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: 2024-02-10T07:40:28.975 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [----------] Global test environment tear-down 2024-02-10T07:40:28.975 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [==========] 1 test from 1 test suite ran. (10750 ms total) 2024-02-10T07:40:28.975 INFO:tasks.workunit.client.0.smithi070.stdout: handler_error: [ PASSED ] 1 test. 2024-02-10T07:40:29.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:40:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:40:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24670 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24808 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24670 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24808 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: pgmap v63: 1156 pgs: 576 unknown, 448 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 0 op/s 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: osdmap e78: 8 total, 8 up, 8 in 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[39923]: osdmap e79: 8 total, 8 up, 8 in 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:29.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:29.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: pgmap v63: 1156 pgs: 576 unknown, 448 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 0 op/s 2024-02-10T07:40:29.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: osdmap e78: 8 total, 8 up, 8 in 2024-02-10T07:40:29.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:29.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:29.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:29.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:29.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:29 smithi070 ceph-mon[29548]: osdmap e79: 8 total, 8 up, 8 in 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24670 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24808 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: pgmap v63: 1156 pgs: 576 unknown, 448 creating+peering, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 857 B/s rd, 0 op/s 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: osdmap e78: 8 total, 8 up, 8 in 2024-02-10T07:40:29.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:29.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:29.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:29.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:29.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:29 smithi155 ceph-mon[32688]: osdmap e79: 8 total, 8 up, 8 in 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (352 ms) 2024-02-10T07:40:29.760 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (27 ms) 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (17 ms) 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (28 ms) 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (10 ms) 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (6 ms) 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (7 ms) 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-10T07:40:29.761 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (5 ms) 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (4 ms) 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (14 ms) 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (8 ms) 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-10T07:40:29.762 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (516 ms total) 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (11653 ms total) 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-02-10T07:40:29.763 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (11642 ms) 2024-02-10T07:40:29.764 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (11642 ms total) 2024-02-10T07:40:29.764 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: 2024-02-10T07:40:29.764 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-02-10T07:40:29.764 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (11642 ms total) 2024-02-10T07:40:29.764 INFO:tasks.workunit.client.0.smithi070.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (53 ms) 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (50 ms) 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (11609 ms) 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-10T07:40:29.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663130+0000 mon.a [INF] from='client.24670 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi070-77283-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663213+0000 mon.a [INF] from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663267+0000 mon.a [INF] from='client.34739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi070-77569-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663341+0000 mon.a [INF] from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi070-77363-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663402+0000 mon.a [INF] from='client.34742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi070-77617-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663450+0000 mon.a [INF] from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi070-77309-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663514+0000 mon.a [INF] from='client.34703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi070-77399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663562+0000 mon.a [INF] from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi070-77348-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663623+0000 mon.a [INF] from='client.34730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi070-77602-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663671+0000 mon.a [INF] from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi070-77613-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663730+0000 mon.a [INF] from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi070-77796-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:29.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663769+0000 mon.a [INF] from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi070-77757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:30.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663807+0000 mon.a [INF] from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77875-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:30.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663850+0000 mon.a [INF] from='client.24808 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi070-78039-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:30.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663925+0000 mon.a [INF] from='client.24829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77851-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.663965+0000 mon.a [INF] from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-78025-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.664001+0000 mon.a [INF] from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.664045+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:27.670444+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:28.324453+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:28.669593+0000 mon.a [INF] from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi070-78461-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:28.669703+0000 mon.a [INF] from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi070-78508-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:29.741578+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:29.749487+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:29.851674+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:30.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:29.856884+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:30.827 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:29.866229+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "te api_cmd_pp: Running main() from gmock_main.cc 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (49 ms) 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (39 ms) 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (12684 ms) 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (12772 ms total) 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (12772 ms total) 2024-02-10T07:40:30.828 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-10T07:40:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[29548]: osdmap e80: 8 total, 8 up, 8 in 2024-02-10T07:40:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:30.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:30 smithi070 ceph-mon[39923]: osdmap e80: 8 total, 8 up, 8 in 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi070-77838-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:30 smithi155 ceph-mon[32688]: osdmap e80: 8 total, 8 up, 8 in 2024-02-10T07:40:31.805 INFO:tasks.workunit.client.0.smithi070.stdout:stprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:31.805 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:29.874684+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:31.805 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.432208+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:40:31.805 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.432762+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:40:31.805 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.433158+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:40:31.805 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.433543+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.433922+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.434310+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.434682+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.435060+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.435436+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.435815+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.807646+0000 mon.a [WRN] Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.808197+0000 mon.a [INF] from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.819998+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/2879039292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.822429+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.822886+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/2888240228' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:31.806 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:30.827288+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:31.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.101449+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-5... 2024-02-10T07:40:31.823 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: listed object 50... 2024-02-10T07:40:31.823 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: listed object 75... 2024-02-10T07:40:31.823 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: listed object 100... 2024-02-10T07:40:31.823 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: listed object 125... 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: saw 150 objects 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_15_[79530]: shutting down. 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_16_[79531]: starting. 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_16_[79531]: added 25 objects... 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_16_[79531]: added half of the objects 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_16_[79531]: added 50 objects... 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_16_[79531]: added 50 objects 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: process_16_[79531]: shutting down. 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.824 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.825 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.825 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.825 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******************************* 2024-02-10T07:40:31.825 INFO:tasks.workunit.client.0.smithi070.stdout: list_parallel: ******* SUCCESS ********** 2024-02-10T07:40:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: pgmap v66: 892 pgs: 104 unknown, 3 creating+activating, 48 creating+peering, 737 active+clean; 607 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 124 KiB/s wr, 349 op/s 2024-02-10T07:40:32.096 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:40:32.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:40:32.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:40:32.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:40:32.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:40:32.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:40:32.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:40:32.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:32.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: pgmap v66: 892 pgs: 104 unknown, 3 creating+activating, 48 creating+peering, 737 active+clean; 607 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 124 KiB/s wr, 349 op/s 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:40:32.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:40:32.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: osdmap e81: 8 total, 8 up, 8 in 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2879039292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2888240228' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:32.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:32.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: osdmap e81: 8 total, 8 up, 8 in 2024-02-10T07:40:32.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2879039292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2888240228' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:32.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:32.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:31 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: pgmap v66: 892 pgs: 104 unknown, 3 creating+activating, 48 creating+peering, 737 active+clean; 607 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 124 KiB/s wr, 349 op/s 2024-02-10T07:40:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:40:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:40:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:40:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:40:32.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:40:32.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:40:32.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:40:32.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:40:32.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:40:32.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:40:32.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:40:32.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:40:32.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:40:32.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:40:32.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:40:32.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:40:32.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:40:32.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:40:32.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:40:32.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:40:32.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:32.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:32.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.260 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: osdmap e81: 8 total, 8 up, 8 in 2024-02-10T07:40:32.260 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2879039292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.260 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.260 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2888240228' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.260 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:32.260 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:32.261 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:31 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:40:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:40:32.816 INFO:tasks.workunit.client.0.smithi070.stdout:LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:32.816 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.335787+0000 mon.a [INF] from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:32.816 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:29.853179+0000 client.admin [INF] onexx 2024-02-10T07:40:32.816 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.336652+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:32.816 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.570507+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.816 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.811995+0000 mon.a [INF] from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.813656+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.813790+0000 mon.a [INF] from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.813969+0000 mon.a [INF] from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.814069+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.819906+0000 mon.a [INF] from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.820206+0000 mon.a [INF] from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.821174+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3147867676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.821707+0000 mon.a [INF] from='client.35108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.824972+0000 mon.a [INF] from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.831604+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.831937+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:32.817 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.841723+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: onexx 2024-02-10T07:40:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: 15.1 deep-scrub starts 2024-02-10T07:40:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: 15.1 deep-scrub ok 2024-02-10T07:40:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: 15.3 deep-scrub starts 2024-02-10T07:40:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: 15.3 deep-scrub ok 2024-02-10T07:40:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: pgmap v68: 852 pgs: 232 unknown, 3 creating+activating, 40 creating+peering, 577 active+clean; 599 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 103 KiB/s wr, 310 op/s 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: osdmap e82: 8 total, 8 up, 8 in 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:33.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3147867676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.35108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:33.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[29548]: from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: onexx 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: 15.1 deep-scrub starts 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: 15.1 deep-scrub ok 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: 15.3 deep-scrub starts 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: 15.3 deep-scrub ok 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: pgmap v68: 852 pgs: 232 unknown, 3 creating+activating, 40 creating+peering, 577 active+clean; 599 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 103 KiB/s wr, 310 op/s 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: osdmap e82: 8 total, 8 up, 8 in 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3147867676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.35108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:33.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:33.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:33.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:33.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:33.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:32 smithi070 ceph-mon[39923]: from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: onexx 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: 15.1 deep-scrub starts 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: 15.1 deep-scrub ok 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: 15.3 deep-scrub starts 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: 15.3 deep-scrub ok 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: pgmap v68: 852 pgs: 232 unknown, 3 creating+activating, 40 creating+peering, 577 active+clean; 599 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 103 KiB/s wr, 310 op/s 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: osdmap e82: 8 total, 8 up, 8 in 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:33.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3147867676' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.35108 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:33.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:32 smithi155 ceph-mon[32688]: from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:34.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.0 deep-scrub starts 2024-02-10T07:40:34.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.0 deep-scrub ok 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.7 deep-scrub starts 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.7 deep-scrub ok 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.6 deep-scrub starts 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.6 deep-scrub ok 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.9 deep-scrub starts 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: 15.9 deep-scrub ok 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: twoxx 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.25216 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: osdmap e83: 8 total, 8 up, 8 in 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:34.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.0 deep-scrub starts 2024-02-10T07:40:34.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.0 deep-scrub ok 2024-02-10T07:40:34.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.7 deep-scrub starts 2024-02-10T07:40:34.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.7 deep-scrub ok 2024-02-10T07:40:34.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.6 deep-scrub starts 2024-02-10T07:40:34.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.6 deep-scrub ok 2024-02-10T07:40:34.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.9 deep-scrub starts 2024-02-10T07:40:34.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: 15.9 deep-scrub ok 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: twoxx 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.25216 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: osdmap e83: 8 total, 8 up, 8 in 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:34.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:34.253 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.253 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:33 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.0 deep-scrub starts 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.0 deep-scrub ok 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.7 deep-scrub starts 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.7 deep-scrub ok 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.6 deep-scrub starts 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.6 deep-scrub ok 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.9 deep-scrub starts 2024-02-10T07:40:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: 15.9 deep-scrub ok 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: twoxx 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35108 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi070-77269-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.25216 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: osdmap e83: 8 total, 8 up, 8 in 2024-02-10T07:40:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3998860341' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:34.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:33 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]: dispatch 2024-02-10T07:40:35.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.4 deep-scrub starts 2024-02-10T07:40:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.4 deep-scrub ok 2024-02-10T07:40:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.2 deep-scrub starts 2024-02-10T07:40:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.2 deep-scrub ok 2024-02-10T07:40:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.5 deep-scrub starts 2024-02-10T07:40:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.5 deep-scrub ok 2024-02-10T07:40:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.8 deep-scrub starts 2024-02-10T07:40:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: 15.8 deep-scrub ok 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: pgmap v71: 884 pgs: 392 unknown, 3 creating+activating, 8 creating+peering, 481 active+clean; 586 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: osdmap e84: 8 total, 8 up, 8 in 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:35.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1540803862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:35.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35294 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:35.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.4 deep-scrub starts 2024-02-10T07:40:35.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.4 deep-scrub ok 2024-02-10T07:40:35.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.2 deep-scrub starts 2024-02-10T07:40:35.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.2 deep-scrub ok 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.5 deep-scrub starts 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.5 deep-scrub ok 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.8 deep-scrub starts 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: 15.8 deep-scrub ok 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: pgmap v71: 884 pgs: 392 unknown, 3 creating+activating, 8 creating+peering, 481 active+clean; 586 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: osdmap e84: 8 total, 8 up, 8 in 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1540803862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:35.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35294 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:35.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.252 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.252 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.252 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.252 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.252 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.4 deep-scrub starts 2024-02-10T07:40:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.4 deep-scrub ok 2024-02-10T07:40:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.2 deep-scrub starts 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.2 deep-scrub ok 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.5 deep-scrub starts 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.5 deep-scrub ok 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.8 deep-scrub starts 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: 15.8 deep-scrub ok 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: pgmap v71: 884 pgs: 392 unknown, 3 creating+activating, 8 creating+peering, 481 active+clean; 586 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T07:40:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: osdmap e84: 8 total, 8 up, 8 in 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1540803862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35294 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:35.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:35.822 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:31.889527+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:35.822 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:32.575855+0000 mon.a [INF] from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.822 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:32.576549+0000 mon.a [INF] from='client.25216 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.822 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:33.812997+0000 mon.a [INF] from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:35.822 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:33.817380+0000 mon.a [INF] from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T07:40:35.822 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:33.822585+0000 mon.a [INF] from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.065345+0000 mon.a [INF] from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi070-77613-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.065413+0000 mon.a [INF] from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi070-77309-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.065458+0000 mon.a [INF] from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.065497+0000 mon.a [INF] from='client.24883 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi070-78461-1"}]': finished 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.065531+0000 mon.a [INF] from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.065572+0000 mon.a [INF] from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.070404+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.071125+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.072176+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.073393+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.073859+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:35.823 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.074406+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.075003+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.078257+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1540803862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.082123+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.088458+0000 mon.a [INF] from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.095399+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.102373+0000 mon.a [INF] from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.102915+0000 mon.a [INF] from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.103384+0000 mon.a [INF] from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.103782+0000 mon.a [INF] from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.104220+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:35.824 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.104554+0000 mon.a [INF] from='client.35294 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.105954+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-doma delete_pools_parallel: process_1_[78305]: starting. 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_1_[78305]: creating pool ceph_test_rados_delete_pools_parallel.smithi070-78155 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_1_[78305]: created object 0... 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_1_[78305]: created object 25... 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_1_[78305]: created object 49... 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_1_[78305]: finishing. 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_1_[78305]: shutting down. 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_2_[78309]: starting. 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_2_[78309]: deleting pool ceph_test_rados_delete_pools_parallel.smithi070-78155 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_2_[78309]: shutting down. 2024-02-10T07:40:36.094 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: ******************************* 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_3_[79613]: starting. 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_3_[79613]: creating pool ceph_test_rados_delete_pools_parallel.smithi070-78155 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_3_[79613]: created object 0... 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_3_[79613]: created object 25... 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_3_[79613]: created object 49... 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_3_[79613]: finishing. 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_3_[79613]: shutting down. 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: ******************************* 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_5_[79615]: starting. 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_5_[79615]: listing objects. 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_5_[79615]: listed object 0... 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_5_[79615]: listed object 25... 2024-02-10T07:40:36.095 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_5_[79615]: saw 50 objects 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_5_[79615]: shutting down. 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: ******************************* 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_4_[79614]: starting. 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_4_[79614]: deleting pool ceph_test_rados_delete_pools_parallel.smithi070-78155 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: process_4_[79614]: shutting down. 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: ******************************* 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: ******************************* 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: cls: Running main() from gmock_main.cc 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [----------] Global test environment set-up. 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-02-10T07:40:36.096 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [ OK ] NeoRadosCls.DNE (11614 ms) 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (6315 ms) 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [----------] 2 tests from NeoRadosCls (17929 ms total) 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [----------] Global test environment tear-down 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [==========] 2 tests from 1 test suite ran. (17929 ms total) 2024-02-10T07:40:36.097 INFO:tasks.workunit.client.0.smithi070.stdout: cls: [ PASSED ] 2 tests. 2024-02-10T07:40:36.365 INFO:tasks.workunit.client.0.smithi070.stdout:in=osd"]}]: dispatch 2024-02-10T07:40:36.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.106154+0000 mon.a [INF] from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:36.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.106759+0000 mon.a [INF] from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:36.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.107129+0000 mon.a [INF] from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:36.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.107569+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.107971+0000 mon.a [INF] from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.108252+0000 mon.a [INF] from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.114376+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.121169+0000 mon.a [INF] from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.128849+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:34.129611+0000 mon.a [INF] from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.068967+0000 mon.a [INF] from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.069048+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.069125+0000 mon.a [INF] from='client.35294 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.069204+0000 mon.a [INF] from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.069260+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.069315+0000 mon.a [INF] from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.069394+0000 mon.a [INF] from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.069480+0000 mon.a [INF] from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.073956+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.074667+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.074946+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.075254+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.075851+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.076244+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.077389+0000 mon.b [INF] from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.822084+0000 mon.c [INF] from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.092973+0000 mon.a [INF] from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.093195+0000 mon.a [INF] from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.096003+0000 mon.a [INF] from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.35294 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:36.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.35294 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:36.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: osdmap e85: 8 total, 8 up, 8 in 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:36.747 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: osdmap e86: 8 total, 8 up, 8 in 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: osdmap e85: 8 total, 8 up, 8 in 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:36.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:36.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:36.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:36.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: osdmap e86: 8 total, 8 up, 8 in 2024-02-10T07:40:36.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:36.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:36.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:36 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.35294 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi070-77363-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi070-77283-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi070-78461-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/650011746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1788153004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:36.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:36.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:36.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1195986569' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:36.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:36.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: osdmap e85: 8 total, 8 up, 8 in 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]: dispatch 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: osdmap e86: 8 total, 8 up, 8 in 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:36.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:36 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:38.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[29548]: pgmap v74: 772 pgs: 128 unknown, 96 creating+peering, 548 active+clean; 145 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 229 op/s 2024-02-10T07:40:38.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[29548]: threexx 2024-02-10T07:40:38.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[29548]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:38.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:38.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:38.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[39923]: pgmap v74: 772 pgs: 128 unknown, 96 creating+peering, 548 active+clean; 145 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 229 op/s 2024-02-10T07:40:38.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[39923]: threexx 2024-02-10T07:40:38.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[39923]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:38.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:38.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:38 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:38.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:38 smithi155 ceph-mon[32688]: pgmap v74: 772 pgs: 128 unknown, 96 creating+peering, 548 active+clean; 145 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 229 op/s 2024-02-10T07:40:38.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:38 smithi155 ceph-mon[32688]: threexx 2024-02-10T07:40:38.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:38 smithi155 ceph-mon[32688]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:38.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:38.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:38 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:39.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:40:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:40:39.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.096216+0 cmd: Running main() from gmock_main.cc 2024-02-10T07:40:39.347 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [----------] Global test environment set-up. 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (10103 ms) 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (3509 ms) 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (7522 ms) 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [----------] 3 tests from NeoRadosCmd (21134 ms total) 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [----------] Global test environment tear-down 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [==========] 3 tests from 1 test suite ran. (21134 ms total) 2024-02-10T07:40:39.348 INFO:tasks.workunit.client.0.smithi070.stdout: cmd: [ PASSED ] 3 tests. 2024-02-10T07:40:39.350 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_1_[78301]: starting. 2024-02-10T07:40:39.350 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_1_[78301]: creating pool ceph_test_rados_open_pools_parallel.smithi070-78223 2024-02-10T07:40:39.350 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_1_[78301]: created object 0... 2024-02-10T07:40:39.350 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_1_[78301]: created object 25... 2024-02-10T07:40:39.350 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_1_[78301]: created object 49... 2024-02-10T07:40:39.350 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_1_[78301]: finishing. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_1_[78301]: shutting down. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_2_[78307]: starting. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_2_[78307]: rados_pool_create. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_2_[78307]: rados_ioctx_create. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_2_[78307]: shutting down. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: ******************************* 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_3_[79770]: starting. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_3_[79770]: creating pool ceph_test_rados_open_pools_parallel.smithi070-78223 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_3_[79770]: created object 0... 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_3_[79770]: created object 25... 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_3_[79770]: created object 49... 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_3_[79770]: finishing. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_3_[79770]: shutting down. 2024-02-10T07:40:39.351 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: ******************************* 2024-02-10T07:40:39.352 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_4_[79771]: starting. 2024-02-10T07:40:39.352 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_4_[79771]: rados_pool_create. 2024-02-10T07:40:39.352 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_4_[79771]: rados_ioctx_create. 2024-02-10T07:40:39.352 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: process_4_[79771]: shutting down. 2024-02-10T07:40:39.352 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: ******************************* 2024-02-10T07:40:39.352 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: ******************************* 2024-02-10T07:40:39.352 INFO:tasks.workunit.client.0.smithi070.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: seed 77318 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (66 ms) 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (1188 ms) 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-10T07:40:39.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (10 ms) 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (11 ms) 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (56 ms) 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (46 ms) 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (6 ms) 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (4 ms) 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (12 ms) 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-10T07:40:39.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (5 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (8 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (6 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (7 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (15 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (6 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (7 ms) 2024-02-10T07:40:39.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (5 ms) 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (4 ms) 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (1466 ms total) 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (3321 ms) 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (21 ms) 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (6 ms) 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-10T07:40:39.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (4 ms) 2024-02-10T07:40:39.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-10T07:40:39.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (4 ms) 2024-02-10T07:40:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2494081265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: osdmap e87: 8 total, 8 up, 8 in 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.35309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T07:40:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: fourxx 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2494081265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: osdmap e87: 8 total, 8 up, 8 in 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.35309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: fourxx 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T07:40:39.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:39 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi070-78508-1"}]': finished 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-4", "overlaypool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]': finished 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2494081265' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: osdmap e87: 8 total, 8 up, 8 in 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.35309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2582757084' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: fourxx 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T07:40:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:39 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi070-77399-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: pgmap v77: 836 pgs: 224 unknown, 64 creating+peering, 548 active+clean; 145 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 341 op/s 2024-02-10T07:40:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: osdmap e88: 8 total, 8 up, 8 in 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]': finished 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.35309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: osdmap e89: 8 total, 8 up, 8 in 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2896051737' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2674353525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]: dispatch 2024-02-10T07:40:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: pgmap v77: 836 pgs: 224 unknown, 64 creating+peering, 548 active+clean; 145 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 341 op/s 2024-02-10T07:40:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: osdmap e88: 8 total, 8 up, 8 in 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]': finished 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.35309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: osdmap e89: 8 total, 8 up, 8 in 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2896051737' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2674353525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.35369 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25354 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:40.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.35369 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25354 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:40.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:40 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: pgmap v77: 836 pgs: 224 unknown, 64 creating+peering, 548 active+clean; 145 MiB data, 711 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 341 op/s 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: osdmap e88: 8 total, 8 up, 8 in 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-4-cache", "mode": "writeback"}]': finished 2024-02-10T07:40:41.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.35309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi070-77269-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi070-78508-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.35207 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: osdmap e89: 8 total, 8 up, 8 in 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2896051737' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2674353525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.35369 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25354 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:41.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:40 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:41.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi070-77838-1"}]: dispatch 2024-02-10T07:40:41.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.096461+0000 mon.a [INF] from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi070-77309-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:41.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.096632+0000 mon.a [INF] from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi070-77613-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:41.342 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.096714+0000 mon.a [INF] from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi070-77318-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.096796+0000 mon.a [INF] from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi070-77757-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.105486+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi070-77283-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.105707+0000 mon.a [INF] from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi070-77363-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.106196+0000 mon.a [INF] from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi070-78461-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:35.822238+0000 client.admin [INF] threexx 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.363464+0000 mon.a [WRN] Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: got: 2024-02-10T07:40:36.620454+0000 mon.a [INF] from='client.35207 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (11589 ms) 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (23301 ms total) 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (23301 ms total) 2024-02-10T07:40:41.343 INFO:tasks.workunit.client.0.smithi070.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-10T07:40:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: pgmap v80: 924 pgs: 160 unknown, 146 creating+peering, 618 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.35369 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25354 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]': finished 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: osdmap e90: 8 total, 8 up, 8 in 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]: dispatch 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: pgmap v80: 924 pgs: 160 unknown, 146 creating+peering, 618 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.35369 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25354 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]': finished 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: osdmap e90: 8 total, 8 up, 8 in 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]: dispatch 2024-02-10T07:40:41.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:41.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:41.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:40:41.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:41.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:41.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:41.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:40:41.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:41 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: pgmap v80: 924 pgs: 160 unknown, 146 creating+peering, 618 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.35369 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25354 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-4"}]': finished 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2073097772' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: osdmap e90: 8 total, 8 up, 8 in 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:40:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:41 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi070-78508-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:42.174 INFO:tasks.workunit.client.0.smithi070.stdout: list: Running main() from gmock_main.cc 2024-02-10T07:40:42.174 INFO:tasks.workunit.client.0.smithi070.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-02-10T07:40:42.174 INFO:tasks.workunit.client.0.smithi070.stdout: list: [----------] Global test environment set-up. 2024-02-10T07:40:42.174 INFO:tasks.workunit.client.0.smithi070.stdout: list: [----------] 3 tests from NeoradosList 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [ OK ] NeoradosList.ListObjects (11485 ms) 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [ OK ] NeoradosList.ListObjectsNS (6338 ms) 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [ OK ] NeoradosList.ListObjectsMany (6083 ms) 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [----------] 3 tests from NeoradosList (23906 ms total) 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [----------] Global test environment tear-down 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [==========] 3 tests from 1 test suite ran. (23906 ms total) 2024-02-10T07:40:42.175 INFO:tasks.workunit.client.0.smithi070.stdout: list: [ PASSED ] 3 tests. 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (5 ms) 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-10T07:40:42.190 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (8 ms) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (8 ms) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (6 ms) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (3415 ms total) 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: 2024-02-10T07:40:42.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24323 ms total) 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: seed 77757 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-10T07:40:42.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (358 ms) 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (28 ms) 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5 ms) 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (16 ms) 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (408 ms total) 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (3344 ms) 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-10T07:40:42.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (4 ms) 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (7 ms) 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (3355 ms total) 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (24170 ms total) 2024-02-10T07:40:42.194 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-10T07:40:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:40:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:40:43.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: Running main() from gmock_main.cc 2024-02-10T07:40:43.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-02-10T07:40:43.191 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [----------] Global test environment set-up. 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStat.Stat (357 ms) 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (8 ms) 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStat.StatNS (21 ms) 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (9 ms) 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [----------] 5 tests from LibRadosStat (396 ms total) 2024-02-10T07:40:43.192 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1077 ms) 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (2251 ms) 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (2 ms) 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (13 ms) 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (3343 ms total) 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [----------] Global test environment tear-down 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (25217 ms total) 2024-02-10T07:40:43.193 INFO:tasks.workunit.client.0.smithi070.stdout: api_stat: [ PASSED ] 9 tests. 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: Running main() from gmock_main.cc 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [----------] Global test environment set-up. 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (381 ms) 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-10T07:40:43.197 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: no timeout :/ 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: no timeout :/ 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: no timeout :/ 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: no timeout :/ 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: no timeout :/ 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (56 ms) 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (8 ms) 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.Checksum (4 ms) 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (4 ms) 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-10T07:40:43.198 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (6 ms) 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (6 ms) 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.RmXattr (14 ms) 2024-02-10T07:40:43.199 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIo.XattrIter (7 ms) 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [----------] 14 tests from LibRadosIo (500 ms total) 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (3339 ms) 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (11 ms) 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (6 ms) 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (7 ms) 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-10T07:40:43.200 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (4 ms) 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [----------] 10 tests from LibRadosIoEC (3393 ms total) 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [----------] Global test environment tear-down 2024-02-10T07:40:43.201 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [==========] 24 tests from 2 test suites ran. (25329 ms total) 2024-02-10T07:40:43.202 INFO:tasks.workunit.client.0.smithi070.stdout: api_io: [ PASSED ] 24 tests. 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]': finished 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: osdmap e91: 8 total, 8 up, 8 in 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: pool 'PoolQuotaPP_smithi070-77283-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:43.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2173217448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/591659480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: osdmap e92: 8 total, 8 up, 8 in 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:43.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[39923]: from='client.35393 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:43.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]': finished 2024-02-10T07:40:43.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: osdmap e91: 8 total, 8 up, 8 in 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: pool 'PoolQuotaPP_smithi070-77283-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T07:40:43.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.247 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2173217448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/591659480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: osdmap e92: 8 total, 8 up, 8 in 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:43.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:42 smithi070 ceph-mon[29548]: from='client.35393 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25081 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-4", "tierpool": "test-rados-api-smithi070-77569-4-cache"}]': finished 2024-02-10T07:40:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77838-6", "pg_num": 4}]': finished 2024-02-10T07:40:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2609302209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3786187134' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: osdmap e91: 8 total, 8 up, 8 in 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]: dispatch 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: pool 'PoolQuotaPP_smithi070-77283-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi070-77796-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25282 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi070-77318-22"}]': finished 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi070-77757-7"}]': finished 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2814073105' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3545077434' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2173217448' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/591659480' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:43.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: osdmap e92: 8 total, 8 up, 8 in 2024-02-10T07:40:43.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:43.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]: dispatch 2024-02-10T07:40:43.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]: dispatch 2024-02-10T07:40:43.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:43.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:42 smithi155 ceph-mon[32688]: from='client.35393 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.191 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: Running main() from gmock_main.cc 2024-02-10T07:40:44.191 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-02-10T07:40:44.191 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [----------] Global test environment set-up. 2024-02-10T07:40:44.191 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-02-10T07:40:44.191 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (15822 ms) 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (10116 ms) 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (25938 ms total) 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [----------] Global test environment tear-down 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (25938 ms total) 2024-02-10T07:40:44.192 INFO:tasks.workunit.client.0.smithi070.stdout: ec_io: [ PASSED ] 2 tests. 2024-02-10T07:40:44.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: pgmap v83: 840 pgs: 204 unknown, 56 creating+peering, 580 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]': finished 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35393 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]: dispatch 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: osdmap e93: 8 total, 8 up, 8 in 2024-02-10T07:40:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/987100169' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]: dispatch 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.35411 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: pgmap v83: 840 pgs: 204 unknown, 56 creating+peering, 580 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]': finished 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:44.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35393 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]: dispatch 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: osdmap e93: 8 total, 8 up, 8 in 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/987100169' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]: dispatch 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.35411 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: pgmap v83: 840 pgs: 204 unknown, 56 creating+peering, 580 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:40:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "overlaypool": "test-rados-api-smithi070-77838-6"}]': finished 2024-02-10T07:40:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35213 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi070-77613-7"}]': finished 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35225 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi070-77309-16"}]': finished 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35393 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi070-77269-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]: dispatch 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/12763828' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: osdmap e93: 8 total, 8 up, 8 in 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/987100169' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]: dispatch 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]: dispatch 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.35411 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: pgmap v86: 808 pgs: 276 unknown, 16 creating+peering, 516 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]': finished 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.35411 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: osdmap e94: 8 total, 8 up, 8 in 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:45.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: pgmap v86: 808 pgs: 276 unknown, 16 creating+peering, 516 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]': finished 2024-02-10T07:40:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.35411 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: osdmap e94: 8 total, 8 up, 8 in 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: pgmap v86: 808 pgs: 276 unknown, 16 creating+peering, 516 active+clean; 145 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:40:45.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77838-6", "mode": "writeback"}]': finished 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.35303 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi070-78461-2"}]': finished 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.35411 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: osdmap e94: 8 total, 8 up, 8 in 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:45.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify_test_cb 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (367 ms) 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94479973461984 err -107 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1024 ms) 2024-02-10T07:40:46.262 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94479973486240 err -107 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1005 ms) 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 343597383681 cookie 94479973506208 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (5 ms) 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 343597383681 cookie 94479973509424 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (5 ms) 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 343597383682 cookie 94479973509424 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (4 ms) 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 343597383683 cookie 94479973509424 2024-02-10T07:40:46.263 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 343597383683 cookie 94479973534368 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (7 ms) 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 343597383680 cookie 94479973509424 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 347892350977 cookie 94479973509424 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3004 ms) 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94479973571456 err -107 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 369367187457 cookie 94479973571456 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (4010 ms) 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94479973571456 err -107 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-10T07:40:46.264 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (84 ms) 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9515 ms total) 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: watch_notify_test_cb 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1087 ms) 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1087 ms total) 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-10T07:40:46.265 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (28236 ms total) 2024-02-10T07:40:46.266 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-10T07:40:46.269 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: seed 77399 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (393 ms) 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (8 ms) 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1007 ms) 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1004 ms) 2024-02-10T07:40:46.270 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (3 ms) 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (5 ms) 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (2 ms) 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2426 ms total) 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (2361 ms) 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-10T07:40:46.271 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (4 ms) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1005 ms) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1004 ms) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (4390 ms total) 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: 2024-02-10T07:40:46.272 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-10T07:40:46.273 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (28358 ms total) 2024-02-10T07:40:46.273 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-10T07:40:46.281 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: Running main() from gmock_main.cc 2024-02-10T07:40:46.281 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-10T07:40:46.281 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [----------] Global test environment set-up. 2024-02-10T07:40:46.281 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (297 ms) 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.LockShared (89 ms) 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1010 ms) 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1004 ms) 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (3 ms) 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.Unlock (4 ms) 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-10T07:40:46.282 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (5 ms) 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [----------] 8 tests from LibRadosLock (2415 ms total) 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (2351 ms) 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (12 ms) 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1005 ms) 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1005 ms) 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-10T07:40:46.283 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (5 ms) 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (4389 ms total) 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [----------] Global test environment tear-down 2024-02-10T07:40:46.284 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (28383 ms total) 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: Running main() from gmock_main.cc 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [----------] Global test environment set-up. 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ OK ] NeoRadosPools.PoolList (9725 ms) 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (3814 ms) 2024-02-10T07:40:46.285 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (4260 ms) 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (6079 ms) 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2030 ms) 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2075 ms) 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [----------] 6 tests from NeoRadosPools (27983 ms total) 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [----------] Global test environment tear-down 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [==========] 6 tests from 1 test suite ran. (27983 ms total) 2024-02-10T07:40:46.286 INFO:tasks.workunit.client.0.smithi070.stdout: pool: [ PASSED ] 6 tests. 2024-02-10T07:40:46.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:46.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:46.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:46.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:46.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:46.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: osdmap e95: 8 total, 8 up, 8 in 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2613932750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi070-77602-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:46.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: osdmap e95: 8 total, 8 up, 8 in 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2613932750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi070-77602-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/829533758' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1966709041' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3809842820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2390763551' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: osdmap e95: 8 total, 8 up, 8 in 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2613932750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi070-77602-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]: dispatch 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]: dispatch 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]: dispatch 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]: dispatch 2024-02-10T07:40:46.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.357 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-10T07:40:47.357 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-10T07:40:47.357 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-10T07:40:47.357 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: notify 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (2141 ms) 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (17 ms) 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (2158 ms total) 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: notify 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (141 ms) 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: notify 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3827 ms) 2024-02-10T07:40:47.358 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (10 ms) 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (5 ms) 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403413920 notify_id 403726925829 notifier_gid 25363 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403413920 notify_id 403726925830 notifier_gid 25363 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (4 ms) 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403413920 notify_id 403726925827 notifier_gid 25363 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (4 ms) 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403418224 notify_id 403726925831 notifier_gid 25363 2024-02-10T07:40:47.359 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403413920 notify_id 403726925832 notifier_gid 25363 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (4 ms) 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403416096 notify_id 403726925833 notifier_gid 25363 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (4 ms) 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: trying... 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403416096 notify_id 403726925828 notifier_gid 25363 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: timed out 2024-02-10T07:40:47.360 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: flushing 2024-02-10T07:40:47.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_waapi_service: 416 unknown 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 132 active+clean 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 96 creating+peering 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: io: 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: client: 639 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4184 ms) 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ OK ] LibRadosService.Status (20029 ms) 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [----------] 4 tests from LibRadosService (29308 ms total) 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [----------] Global test environment tear-down 2024-02-10T07:40:47.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [==========] 4 tests from 1 test suite ran. (29308 ms total) 2024-02-10T07:40:47.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service: [ PASSED ] 4 tests. 2024-02-10T07:40:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: pgmap v89: 744 pgs: 160 unknown, 96 creating+peering, 488 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-10T07:40:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2613932750' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi070-77602-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: osdmap e96: 8 total, 8 up, 8 in 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1278557433' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3846973738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.25486 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.25489 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: pgmap v89: 744 pgs: 160 unknown, 96 creating+peering, 488 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-10T07:40:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2613932750' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi070-77602-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: osdmap e96: 8 total, 8 up, 8 in 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1278557433' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3846973738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.25486 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.25489 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:47 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: pgmap v89: 744 pgs: 160 unknown, 96 creating+peering, 488 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2613932750' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi070-77602-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.25393 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi070-77796-12"}]': finished 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.25315 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi070-77399-10"}]': finished 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.25366 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi070-78508-2"}]': finished 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.35279 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi070-77363-10"}]': finished 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: osdmap e96: 8 total, 8 up, 8 in 2024-02-10T07:40:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1278557433' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3846973738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.25486 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.25489 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:47 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:40:48.386 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-10T07:40:48.386 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-10T07:40:48.386 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-10T07:40:48.386 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-10T07:40:48.386 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5043 ms) 2024-02-10T07:40:48.386 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-10T07:40:48.386 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (61 ms) 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20056 ms) 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: attempt 0 of 20 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: failed to deregister: 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: { 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "epoch": 18, 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "modified": "2024-02-10T07:40:43.785050+0000", 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "services": { 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "laundry": { 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "daemons": { 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "summary": "", 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "close-test-pid77949": { 2024-02-10T07:40:48.387 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "start_epoch": 18, 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "start_stamp": "2024-02-10T07:40:43.292967+0000", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "gid": 35438, 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "addr": "172.21.15.70:0/3933344421", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "metadata": { 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "arch": "x86_64", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_release": "squid", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_version_short": "19.0.0-1361-g290212a1", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro": "centos", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro_version": "9", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "foo": "bar", 2024-02-10T07:40:48.388 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "hostname": "smithi070", 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024", 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "kernel_version": "5.14.0-419.el9.x86_64", 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "mem_total_kb": "32488372", 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "os": "Linux", 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "this": "that" 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: }, 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "task_status": {} 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: }, 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "pid77887": { 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "start_epoch": 17, 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "start_stamp": "2024-02-10T07:40:27.386108+0000", 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "gid": 14790, 2024-02-10T07:40:48.389 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "addr": "172.21.15.70:0/102481230", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "metadata": { 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "arch": "x86_64", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_release": "squid", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_version_short": "19.0.0-1361-g290212a1", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro": "centos", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro_version": "9", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "foo": "bar", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "hostname": "smithi070", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "kernel_version": "5.14.0-419.el9.x86_64", 2024-02-10T07:40:48.390 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "mem_total_kb": "32488372", 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "os": "Linux", 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "this": "that" 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: }, 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "task_status": {} 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: } 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: } 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: }, 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "rgw": { 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "daemons": { 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "summary": "", 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "14481": { 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "start_epoch": 12, 2024-02-10T07:40:48.391 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "start_stamp": "2024-02-10T07:38:05.287558+0000", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "gid": 14481, 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "addr": "172.21.15.70:0/375850049", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "metadata": { 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "arch": "x86_64", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_release": "squid", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "ceph_version_short": "19.0.0-1361-g290212a1", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "container_hostname": "smithi070", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro": "centos", 2024-02-10T07:40:48.392 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "distro_version": "8", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "hostname": "smithi070", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "id": "foo.a", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "kernel_version": "5.14.0-419.el9.x86_64", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "mem_total_kb": "32488372", 2024-02-10T07:40:48.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "num_handles": "1", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "os": "Linux", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "pid": "2", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "realm_id": "", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "realm_name": "", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "zone_id": "e2c2630b-454e-4991-b189-507c4abc2d1d", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "zone_name": "default", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "zonegroup_id": "c15549f1-6b15-43b4-8f2b-f6a2a4d7e2d2", 2024-02-10T07:40:48.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-10T07:40:48.901 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:48.901 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.25486 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.25489 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: osdmap e97: 8 total, 8 up, 8 in 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:40:48.902 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.25486 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.25489 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: osdmap e97: 8 total, 8 up, 8 in 2024-02-10T07:40:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.25486 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi070-77269-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.25489 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi070-78508-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: osdmap e97: 8 total, 8 up, 8 in 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T07:40:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:49.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:40:48 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:40:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: pgmap v92: 776 pgs: 288 unknown, 488 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-10T07:40:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4162108903' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3588773294' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: osdmap e98: 8 total, 8 up, 8 in 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.25507 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.25507 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: osdmap e99: 8 total, 8 up, 8 in 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1973589229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[39923]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: pgmap v92: 776 pgs: 288 unknown, 488 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4162108903' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3588773294' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: osdmap e98: 8 total, 8 up, 8 in 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.25507 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.25507 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:49.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:49.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:49.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: osdmap e99: 8 total, 8 up, 8 in 2024-02-10T07:40:49.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:49.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1973589229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:49.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:49 smithi070 ceph-mon[29548]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: pgmap v92: 776 pgs: 288 unknown, 488 active+clean; 145 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4162108903' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3588773294' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: osdmap e98: 8 total, 8 up, 8 in 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.25507 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi070-78508-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.25507 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: osdmap e99: 8 total, 8 up, 8 in 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1973589229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:49 smithi155 ceph-mon[32688]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]': finished 2024-02-10T07:40:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]': finished 2024-02-10T07:40:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:40:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-7"}]': finished 2024-02-10T07:40:52.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:40:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:40:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: pgmap v95: 624 pgs: 32 creating+peering, 104 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3373273842' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2226794105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: osdmap e100: 8 total, 8 up, 8 in 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25558 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:40:52.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: osdmap e101: 8 total, 8 up, 8 in 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2072961507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[39923]: from='client.35546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: pgmap v95: 624 pgs: 32 creating+peering, 104 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3373273842' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2226794105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:40:52.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: osdmap e100: 8 total, 8 up, 8 in 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25558 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: osdmap e101: 8 total, 8 up, 8 in 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2072961507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:51 smithi070 ceph-mon[29548]: from='client.35546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: pgmap v95: 624 pgs: 32 creating+peering, 104 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi070-77269-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3373273842' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2226794105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: osdmap e100: 8 total, 8 up, 8 in 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25558 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:40:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25558 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi070-77617-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: osdmap e101: 8 total, 8 up, 8 in 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2072961507' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]: dispatch 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:51 smithi155 ceph-mon[32688]: from='client.35546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.412 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: Running main() from gmock_main.cc 2024-02-10T07:40:53.412 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-02-10T07:40:53.412 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [----------] Global test environment set-up. 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (18851 ms) 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (9152 ms) 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (7148 ms) 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [----------] 3 tests from NeoradosECList (35151 ms total) 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [----------] Global test environment tear-down 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (35151 ms total) 2024-02-10T07:40:53.413 INFO:tasks.workunit.client.0.smithi070.stdout: ec_list: [ PASSED ] 3 tests. 2024-02-10T07:40:53.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:53.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: pgmap v98: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T07:40:53.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:53.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]': finished 2024-02-10T07:40:53.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:53.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.35546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: osdmap e102: 8 total, 8 up, 8 in 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2943774290' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi070-77269-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1855056470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[39923]: from='client.25585 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:53.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: pgmap v98: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]': finished 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.35546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: osdmap e102: 8 total, 8 up, 8 in 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2943774290' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi070-77269-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1855056470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:53.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:53 smithi070 ceph-mon[29548]: from='client.25585 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: }, 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: "task_status": {} 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: } 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: } 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: } 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: } 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: } 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: attempt 1 of 20 2024-02-10T07:40:53.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10232 ms) 2024-02-10T07:40:53.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (35392 ms total) 2024-02-10T07:40:53.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: 2024-02-10T07:40:53.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-10T07:40:53.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (35392 ms total) 2024-02-10T07:40:53.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-10T07:40:53.509 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:53.509 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: pgmap v98: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T07:40:53.509 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:53.509 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-7"}]': finished 2024-02-10T07:40:53.509 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:53.509 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.35546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:53.509 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: osdmap e102: 8 total, 8 up, 8 in 2024-02-10T07:40:53.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2943774290' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi070-77269-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2920716593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:53.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1855056470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:53.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:53.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]: dispatch 2024-02-10T07:40:53.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:53 smithi155 ceph-mon[32688]: from='client.25585 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:54.431 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: Running main() from gmock_main.cc 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [----------] Global test environment set-up. 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (12507 ms) 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (5271 ms) 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (4543 ms) 2024-02-10T07:40:54.432 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (2550 ms) 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3084 ms) 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3040 ms) 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (5119 ms) 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (36114 ms total) 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [----------] Global test environment tear-down 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (36114 ms total) 2024-02-10T07:40:54.433 INFO:tasks.workunit.client.0.smithi070.stdout: write_operations: [ PASSED ] 7 tests. 2024-02-10T07:40:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2943774290' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi070-77269-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='client.25585 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: osdmap e103: 8 total, 8 up, 8 in 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1163245520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='client.35579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1782975795' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2943774290' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi070-77269-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='client.25585 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: osdmap e103: 8 total, 8 up, 8 in 2024-02-10T07:40:54.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1163245520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:54.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='client.35579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:54.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1782975795' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:54.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:54.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2943774290' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi070-77269-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi070-78508-3"}]': finished 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='client.25585 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: osdmap e103: 8 total, 8 up, 8 in 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1163245520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='client.35579 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1782975795' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T07:40:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:55.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[39923]: pgmap v101: 808 pgs: 320 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:40:55.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[39923]: from='client.35579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:55.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[39923]: osdmap e104: 8 total, 8 up, 8 in 2024-02-10T07:40:55.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:55.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[29548]: pgmap v101: 808 pgs: 320 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:40:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[29548]: from='client.35579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[29548]: osdmap e104: 8 total, 8 up, 8 in 2024-02-10T07:40:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:55 smithi155 ceph-mon[32688]: pgmap v101: 808 pgs: 320 unknown, 488 active+clean; 145 MiB data, 765 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:40:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:55 smithi155 ceph-mon[32688]: from='client.35579 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi070-77283-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:55.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:55 smithi155 ceph-mon[32688]: osdmap e104: 8 total, 8 up, 8 in 2024-02-10T07:40:55.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:55.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:40:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: osdmap e105: 8 total, 8 up, 8 in 2024-02-10T07:40:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/525487238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1865073430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.25615 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: osdmap e105: 8 total, 8 up, 8 in 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/525487238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1865073430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.25615 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]: dispatch 2024-02-10T07:40:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:56 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: osdmap e105: 8 total, 8 up, 8 in 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/525487238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.35591 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1865073430' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.25615 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]: dispatch 2024-02-10T07:40:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:56 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]: dispatch 2024-02-10T07:40:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: pgmap v104: 648 pgs: 128 unknown, 32 creating+peering, 488 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:40:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25615 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]': finished 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: osdmap e106: 8 total, 8 up, 8 in 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775059643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25624 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/52340061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:40:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]': finished 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: pgmap v104: 648 pgs: 128 unknown, 32 creating+peering, 488 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25615 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]': finished 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: osdmap e106: 8 total, 8 up, 8 in 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775059643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25624 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/52340061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]': finished 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25624 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:40:57.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: osdmap e107: 8 total, 8 up, 8 in 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25624 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: osdmap e107: 8 total, 8 up, 8 in 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:40:57.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:57 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: pgmap v104: 648 pgs: 128 unknown, 32 creating+peering, 488 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:40:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.35591 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77901-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25615 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi070-77269-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4"}]': finished 2024-02-10T07:40:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: osdmap e106: 8 total, 8 up, 8 in 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775059643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25624 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/52340061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi070-77838-4", "tierpool": "test-rados-api-smithi070-77838-6"}]': finished 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25624 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi070-77283-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:58.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:40:58.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:40:58.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/277224952' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: osdmap e107: 8 total, 8 up, 8 in 2024-02-10T07:40:58.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:40:58.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:57 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-10T07:40:58.292 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (11679 ms) 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6340 ms) 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (5577 ms) 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (4600 ms) 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (5067 ms) 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3085 ms) 2024-02-10T07:40:58.293 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-10T07:40:58.294 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3870 ms) 2024-02-10T07:40:58.294 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (40218 ms total) 2024-02-10T07:40:58.294 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: 2024-02-10T07:40:58.294 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-10T07:40:58.294 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (40219 ms total) 2024-02-10T07:40:58.294 INFO:tasks.workunit.client.0.smithi070.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]': finished 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: osdmap e108: 8 total, 8 up, 8 in 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]: dispatch 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]: dispatch 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1341992091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.903 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-mon[32688]: from='client.25642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]': finished 2024-02-10T07:40:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:40:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: osdmap e108: 8 total, 8 up, 8 in 2024-02-10T07:40:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]: dispatch 2024-02-10T07:40:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]: dispatch 2024-02-10T07:40:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1341992091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[29548]: from='client.25642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-9"}]': finished 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: from='client.25363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi070-77838-6", "pool2": "test-rados-api-smithi070-77838-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: osdmap e108: 8 total, 8 up, 8 in 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]: dispatch 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]: dispatch 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1341992091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:58 smithi070 ceph-mon[39923]: from='client.25642 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:59.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:40:58 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout:tch_notify_pp: flushed 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: trying... 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403416096 notify_id 416611827717 notifier_gid 25363 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: timed out 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: flushing 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: flushed 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3004 ms) 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: List watches 2024-02-10T07:40:59.286 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: notify2 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403416096 notify_id 429496729606 notifier_gid 25363 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: notify2 done 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: watch_check 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: unwatch2 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: flushing 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: done 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3039 ms) 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: List watches 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: notify2 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: handle_notify cookie 94477403416096 notify_id 438086664202 notifier_gid 25363 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: notify2 done 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: watch_check 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: unwatch2 2024-02-10T07:40:59.287 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: flushing 2024-02-10T07:40:59.288 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: done 2024-02-10T07:40:59.288 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3006 ms) 2024-02-10T07:40:59.288 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16063 ms total) 2024-02-10T07:40:59.288 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: 2024-02-10T07:40:59.288 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-10T07:40:59.288 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (41248 ms total) 2024-02-10T07:40:59.288 INFO:tasks.workunit.client.0.smithi070.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-10T07:40:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: pgmap v107: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:40:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]': finished 2024-02-10T07:40:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: from='client.25642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: osdmap e109: 8 total, 8 up, 8 in 2024-02-10T07:40:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4196524940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[29548]: from='client.25651 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: pgmap v107: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]': finished 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: from='client.25642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: osdmap e109: 8 total, 8 up, 8 in 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4196524940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:40:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:40:59 smithi070 ceph-mon[39923]: from='client.25651 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: pgmap v107: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-9", "mode": "writeback"}]': finished 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: from='client.25642 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi070-77269-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: osdmap e109: 8 total, 8 up, 8 in 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4196524940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:40:59 smithi155 ceph-mon[32688]: from='client.25651 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.25651 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: osdmap e110: 8 total, 8 up, 8 in 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.25651 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: osdmap e110: 8 total, 8 up, 8 in 2024-02-10T07:41:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:41:01.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:00 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.25651 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi070-77283-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: osdmap e110: 8 total, 8 up, 8 in 2024-02-10T07:41:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:41:01.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:00 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]: dispatch 2024-02-10T07:41:02.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:41:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:41:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: pgmap v110: 644 pgs: 64 creating+peering, 64 unknown, 516 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:41:02.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:41:02.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:41:02.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:41:02.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:41:02.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:41:02.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: 15.1 deep-scrub starts 2024-02-10T07:41:02.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: 15.1 deep-scrub ok 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: pgmap v110: 644 pgs: 64 creating+peering, 64 unknown, 516 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:41:02.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:41:02.246 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: 15.1 deep-scrub starts 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: 15.1 deep-scrub ok 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: 15.3 deep-scrub starts 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: 15.3 deep-scrub ok 2024-02-10T07:41:02.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: 15.7 deep-scrub starts 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]': finished 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: osdmap e111: 8 total, 8 up, 8 in 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4007578605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.248 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.25666 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: 15.3 deep-scrub starts 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: 15.3 deep-scrub ok 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: 15.7 deep-scrub starts 2024-02-10T07:41:02.249 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]': finished 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: osdmap e111: 8 total, 8 up, 8 in 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4007578605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.25666 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: pgmap v110: 644 pgs: 64 creating+peering, 64 unknown, 516 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:41:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-02-10T07:41:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.70:0/734328326' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: 15.1 deep-scrub starts 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: 15.1 deep-scrub ok 2024-02-10T07:41:02.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: 15.3 deep-scrub starts 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: 15.3 deep-scrub ok 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: 15.7 deep-scrub starts 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-9"}]': finished 2024-02-10T07:41:02.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: osdmap e111: 8 total, 8 up, 8 in 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4007578605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.25666 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:02.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: 15.7 deep-scrub ok 2024-02-10T07:41:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: 15.6 deep-scrub starts 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: 15.6 deep-scrub ok 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: 15.0 deep-scrub starts 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: 15.0 deep-scrub ok 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: pgmap v113: 580 pgs: 128 unknown, 452 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: from='client.25666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: osdmap e112: 8 total, 8 up, 8 in 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2112158357' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi070-77283-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: 15.7 deep-scrub ok 2024-02-10T07:41:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: 15.6 deep-scrub starts 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: 15.6 deep-scrub ok 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: 15.0 deep-scrub starts 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: 15.0 deep-scrub ok 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: pgmap v113: 580 pgs: 128 unknown, 452 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: from='client.25666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: osdmap e112: 8 total, 8 up, 8 in 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2112158357' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi070-77283-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: 15.7 deep-scrub ok 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: 15.6 deep-scrub starts 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: 15.6 deep-scrub ok 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: 15.0 deep-scrub starts 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: 15.0 deep-scrub ok 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: pgmap v113: 580 pgs: 128 unknown, 452 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: from='client.25666 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi070-77269-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:03.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77564-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:03.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: osdmap e112: 8 total, 8 up, 8 in 2024-02-10T07:41:03.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2112158357' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi070-77283-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:03.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:04.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.9 deep-scrub starts 2024-02-10T07:41:04.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.9 deep-scrub ok 2024-02-10T07:41:04.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.4 deep-scrub starts 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.4 deep-scrub ok 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.2 deep-scrub starts 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.2 deep-scrub ok 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.5 deep-scrub starts 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.5 deep-scrub ok 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2112158357' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi070-77283-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: osdmap e113: 8 total, 8 up, 8 in 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T07:41:04.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.8 deep-scrub starts 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[29548]: 15.8 deep-scrub ok 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.9 deep-scrub starts 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.9 deep-scrub ok 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.4 deep-scrub starts 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.4 deep-scrub ok 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.2 deep-scrub starts 2024-02-10T07:41:04.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.2 deep-scrub ok 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.5 deep-scrub starts 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.5 deep-scrub ok 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2112158357' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi070-77283-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: osdmap e113: 8 total, 8 up, 8 in 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.8 deep-scrub starts 2024-02-10T07:41:04.659 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:04 smithi070 ceph-mon[39923]: 15.8 deep-scrub ok 2024-02-10T07:41:04.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.9 deep-scrub starts 2024-02-10T07:41:04.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.9 deep-scrub ok 2024-02-10T07:41:04.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.4 deep-scrub starts 2024-02-10T07:41:04.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.4 deep-scrub ok 2024-02-10T07:41:04.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.2 deep-scrub starts 2024-02-10T07:41:04.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.2 deep-scrub ok 2024-02-10T07:41:04.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.5 deep-scrub starts 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.5 deep-scrub ok 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi070-77602-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2112158357' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi070-77283-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: osdmap e113: 8 total, 8 up, 8 in 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.8 deep-scrub starts 2024-02-10T07:41:04.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:04 smithi155 ceph-mon[32688]: 15.8 deep-scrub ok 2024-02-10T07:41:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: pgmap v116: 588 pgs: 168 unknown, 420 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]': finished 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: osdmap e114: 8 total, 8 up, 8 in 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/455858377' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: from='client.35696 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: pgmap v116: 588 pgs: 168 unknown, 420 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]': finished 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: osdmap e114: 8 total, 8 up, 8 in 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/455858377' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: from='client.35696 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: pgmap v116: 588 pgs: 168 unknown, 420 active+clean; 145 MiB data, 748 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache", "force_nonempty":""}]': finished 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: osdmap e114: 8 total, 8 up, 8 in 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/455858377' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: from='client.35696 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:05.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:06.311 INFO:tasks.workunit.client.0.smithi070.stdout: misc: Running main() from gmock_main.cc 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [----------] Global test environment set-up. 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.Version (9412 ms) 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (4146 ms) 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.LongName (6034 ms) 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-02-10T07:41:06.312 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4319 ms) 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (4107 ms) 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3006 ms) 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.Exec (3054 ms) 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3082 ms) 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (2857 ms) 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-02-10T07:41:06.313 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3008 ms) 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1994 ms) 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3024 ms) 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [----------] 12 tests from NeoRadosMisc (48043 ms total) 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [----------] Global test environment tear-down 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [==========] 12 tests from 1 test suite ran. (48043 ms total) 2024-02-10T07:41:06.314 INFO:tasks.workunit.client.0.smithi070.stdout: misc: [ PASSED ] 12 tests. 2024-02-10T07:41:06.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.35696 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:06.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:06.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: osdmap e115: 8 total, 8 up, 8 in 2024-02-10T07:41:06.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1156091070' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:06.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]: dispatch 2024-02-10T07:41:06.663 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]: dispatch 2024-02-10T07:41:06.664 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:06.664 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:06.664 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.35696 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: osdmap e115: 8 total, 8 up, 8 in 2024-02-10T07:41:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1156091070' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]: dispatch 2024-02-10T07:41:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]: dispatch 2024-02-10T07:41:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.35696 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi070-77269-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77564-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: osdmap e115: 8 total, 8 up, 8 in 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1156091070' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3475589000' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]: dispatch 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]: dispatch 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:06.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: pgmap v119: 652 pgs: 72 creating+peering, 128 unknown, 452 active+clean; 217 MiB data, 1016 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-10T07:41:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1156091070' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]': finished 2024-02-10T07:41:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:41:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: osdmap e116: 8 total, 8 up, 8 in 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]': finished 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: osdmap e117: 8 total, 8 up, 8 in 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]: dispatch 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1994707781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]: dispatch 2024-02-10T07:41:07.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[39923]: from='client.25726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: pgmap v119: 652 pgs: 72 creating+peering, 128 unknown, 452 active+clean; 217 MiB data, 1016 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1156091070' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]': finished 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: osdmap e116: 8 total, 8 up, 8 in 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]': finished 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: osdmap e117: 8 total, 8 up, 8 in 2024-02-10T07:41:07.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]: dispatch 2024-02-10T07:41:07.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1994707781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:07.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]: dispatch 2024-02-10T07:41:07.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:07 smithi070 ceph-mon[29548]: from='client.25726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:07.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: pgmap v119: 652 pgs: 72 creating+peering, 128 unknown, 452 active+clean; 217 MiB data, 1016 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-10T07:41:07.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1156091070' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:07.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.25672 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77564-10", "tierpool":"test-rados-api-smithi070-77564-10-cache"}]': finished 2024-02-10T07:41:07.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:41:07.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:07.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: osdmap e116: 8 total, 8 up, 8 in 2024-02-10T07:41:07.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:07.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:07.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:07.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-11"}]': finished 2024-02-10T07:41:07.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: osdmap e117: 8 total, 8 up, 8 in 2024-02-10T07:41:07.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]: dispatch 2024-02-10T07:41:07.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1994707781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:07.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]: dispatch 2024-02-10T07:41:07.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:07 smithi155 ceph-mon[32688]: from='client.25726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:08.296 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: Running main() from gmock_main.cc 2024-02-10T07:41:08.296 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-10T07:41:08.296 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [----------] Global test environment set-up. 2024-02-10T07:41:08.296 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-10T07:41:08.296 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-10T07:41:08.296 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolList (10366 ms) 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (4517 ms) 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (5024 ms) 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3557 ms) 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (2790 ms) 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-10T07:41:08.297 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (6127 ms) 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5092 ms) 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4868 ms) 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8003 ms) 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [----------] 9 tests from LibRadosPools (50344 ms total) 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [----------] Global test environment tear-down 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (50345 ms total) 2024-02-10T07:41:08.298 INFO:tasks.workunit.client.0.smithi070.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-10T07:41:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:41:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]': finished 2024-02-10T07:41:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[29548]: from='client.25726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[29548]: osdmap e118: 8 total, 8 up, 8 in 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1695249706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[29548]: from='client.35720 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]': finished 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[39923]: from='client.25726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[39923]: osdmap e118: 8 total, 8 up, 8 in 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1695249706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:08 smithi070 ceph-mon[39923]: from='client.35720 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:41:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-11", "mode": "writeback"}]': finished 2024-02-10T07:41:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-mon[32688]: from='client.25726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi070-77269-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-mon[32688]: osdmap e118: 8 total, 8 up, 8 in 2024-02-10T07:41:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1695249706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-mon[32688]: from='client.35720 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:09.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:41:08 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:41:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[39923]: pgmap v122: 588 pgs: 40 creating+peering, 96 unknown, 452 active+clean; 217 MiB data, 1016 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-10T07:41:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[39923]: from='client.35720 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[39923]: osdmap e119: 8 total, 8 up, 8 in 2024-02-10T07:41:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[29548]: pgmap v122: 588 pgs: 40 creating+peering, 96 unknown, 452 active+clean; 217 MiB data, 1016 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-10T07:41:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[29548]: from='client.35720 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:09.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:09 smithi070 ceph-mon[29548]: osdmap e119: 8 total, 8 up, 8 in 2024-02-10T07:41:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:09 smithi155 ceph-mon[32688]: pgmap v122: 588 pgs: 40 creating+peering, 96 unknown, 452 active+clean; 217 MiB data, 1016 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-10T07:41:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:09 smithi155 ceph-mon[32688]: from='client.35720 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi070-77283-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:09 smithi155 ceph-mon[32688]: osdmap e119: 8 total, 8 up, 8 in 2024-02-10T07:41:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: osdmap e120: 8 total, 8 up, 8 in 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1093512384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[29548]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: osdmap e120: 8 total, 8 up, 8 in 2024-02-10T07:41:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1093512384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:10 smithi070 ceph-mon[39923]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: osdmap e120: 8 total, 8 up, 8 in 2024-02-10T07:41:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1093512384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]: dispatch 2024-02-10T07:41:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:10 smithi155 ceph-mon[32688]: from='client.35735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[29548]: pgmap v125: 524 pgs: 32 creating+peering, 32 unknown, 460 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-02-10T07:41:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]': finished 2024-02-10T07:41:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[29548]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[29548]: osdmap e121: 8 total, 8 up, 8 in 2024-02-10T07:41:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3951677727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi070-77283-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[39923]: pgmap v125: 524 pgs: 32 creating+peering, 32 unknown, 460 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-02-10T07:41:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]': finished 2024-02-10T07:41:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[39923]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[39923]: osdmap e121: 8 total, 8 up, 8 in 2024-02-10T07:41:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3951677727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi070-77283-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:11 smithi155 ceph-mon[32688]: pgmap v125: 524 pgs: 32 creating+peering, 32 unknown, 460 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-02-10T07:41:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:11 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-11"}]': finished 2024-02-10T07:41:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:11 smithi155 ceph-mon[32688]: from='client.35735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi070-77269-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:11 smithi155 ceph-mon[32688]: osdmap e121: 8 total, 8 up, 8 in 2024-02-10T07:41:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3951677727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi070-77283-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:41:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:41:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3951677727' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi070-77283-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[29548]: osdmap e122: 8 total, 8 up, 8 in 2024-02-10T07:41:12.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3951677727' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi070-77283-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:12 smithi070 ceph-mon[39923]: osdmap e122: 8 total, 8 up, 8 in 2024-02-10T07:41:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:12 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3951677727' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi070-77283-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:12 smithi155 ceph-mon[32688]: osdmap e122: 8 total, 8 up, 8 in 2024-02-10T07:41:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: pgmap v128: 556 pgs: 96 unknown, 460 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: osdmap e123: 8 total, 8 up, 8 in 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/393961727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[39923]: from='client.25759 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: pgmap v128: 556 pgs: 96 unknown, 460 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: osdmap e123: 8 total, 8 up, 8 in 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/393961727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:13.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:13 smithi070 ceph-mon[29548]: from='client.25759 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: pgmap v128: 556 pgs: 96 unknown, 460 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-02-10T07:41:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: osdmap e123: 8 total, 8 up, 8 in 2024-02-10T07:41:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:14.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/393961727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:13 smithi155 ceph-mon[32688]: from='client.25759 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.25759 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: osdmap e124: 8 total, 8 up, 8 in 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/700523123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[29548]: from='client.25762 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.25759 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: osdmap e124: 8 total, 8 up, 8 in 2024-02-10T07:41:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:14.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:14.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/700523123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:14.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:14 smithi070 ceph-mon[39923]: from='client.25762 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.25759 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi070-77269-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: osdmap e124: 8 total, 8 up, 8 in 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/125854550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]: dispatch 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/700523123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:14 smithi155 ceph-mon[32688]: from='client.25762 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: Running main() from gmock_main.cc 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: [----------] Global test environment set-up. 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: [----------] 14 tests from NeoRadosIo 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.Limits (10770 ms) 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-02-10T07:41:15.310 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (6624 ms) 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.ReadOp (5035 ms) 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.SparseRead (3561 ms) 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.RoundTrip (4084 ms) 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3055 ms) 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3085 ms) 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (2859 ms) 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-02-10T07:41:15.311 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3016 ms) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.Trunc (2995 ms) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.Remove (3014 ms) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2985 ms) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.RmXattr (2997 ms) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3018 ms) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [----------] 14 tests from NeoRadosIo (57098 ms total) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [----------] Global test environment tear-down 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [==========] 14 tests from 1 test suite ran. (57098 ms total) 2024-02-10T07:41:15.312 INFO:tasks.workunit.client.0.smithi070.stdout: io: [ PASSED ] 14 tests. 2024-02-10T07:41:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[29548]: pgmap v131: 548 pgs: 128 unknown, 420 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[29548]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[29548]: from='client.25762 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[29548]: osdmap e125: 8 total, 8 up, 8 in 2024-02-10T07:41:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[39923]: pgmap v131: 548 pgs: 128 unknown, 420 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[39923]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[39923]: from='client.25762 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[39923]: osdmap e125: 8 total, 8 up, 8 in 2024-02-10T07:41:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:15 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:15 smithi155 ceph-mon[32688]: pgmap v131: 548 pgs: 128 unknown, 420 active+clean; 217 MiB data, 1024 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:15 smithi155 ceph-mon[32688]: from='client.35663 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi070-77602-10"}]': finished 2024-02-10T07:41:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:15 smithi155 ceph-mon[32688]: from='client.25762 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi070-77283-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:15 smithi155 ceph-mon[32688]: osdmap e125: 8 total, 8 up, 8 in 2024-02-10T07:41:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:15 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: pgmap v134: 516 pgs: 32 creating+peering, 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: osdmap e126: 8 total, 8 up, 8 in 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3758978776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[29548]: from='client.35774 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: pgmap v134: 516 pgs: 32 creating+peering, 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: osdmap e126: 8 total, 8 up, 8 in 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3758978776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:16 smithi070 ceph-mon[39923]: from='client.35774 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: pgmap v134: 516 pgs: 32 creating+peering, 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-02-10T07:41:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: osdmap e126: 8 total, 8 up, 8 in 2024-02-10T07:41:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3758978776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:16 smithi155 ceph-mon[32688]: from='client.35774 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: Running main() from gmock_main.cc 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [----------] Global test environment set-up. 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (12520 ms) 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (7037 ms) 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-02-10T07:41:17.540 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (4312 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (4107 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3014 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3054 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (4087 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2854 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3254 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2758 ms) 2024-02-10T07:41:17.541 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (2990 ms) 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3007 ms) 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2998 ms) 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3246 ms) 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (59238 ms total) 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [----------] Global test environment tear-down 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (59238 ms total) 2024-02-10T07:41:17.542 INFO:tasks.workunit.client.0.smithi070.stdout: read_operations: [ PASSED ] 14 tests. 2024-02-10T07:41:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[29548]: from='client.35774 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[29548]: osdmap e127: 8 total, 8 up, 8 in 2024-02-10T07:41:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1774897156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi070-77283-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:17.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[39923]: from='client.35774 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[39923]: osdmap e127: 8 total, 8 up, 8 in 2024-02-10T07:41:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1774897156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi070-77283-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:17 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:17 smithi155 ceph-mon[32688]: from='client.35774 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi070-77269-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:17 smithi155 ceph-mon[32688]: osdmap e127: 8 total, 8 up, 8 in 2024-02-10T07:41:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1774897156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi070-77283-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: pgmap v137: 516 pgs: 32 creating+peering, 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1774897156' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi070-77283-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: osdmap e128: 8 total, 8 up, 8 in 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:18.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: pgmap v137: 516 pgs: 32 creating+peering, 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:41:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1774897156' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi070-77283-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: osdmap e128: 8 total, 8 up, 8 in 2024-02-10T07:41:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: pgmap v137: 516 pgs: 32 creating+peering, 64 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi070-77602-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1774897156' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi070-77283-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: osdmap e128: 8 total, 8 up, 8 in 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:41:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:18 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:19.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:41:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:41:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:41:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: osdmap e129: 8 total, 8 up, 8 in 2024-02-10T07:41:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/644866528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[29548]: from='client.25792 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: osdmap e129: 8 total, 8 up, 8 in 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/644866528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:19 smithi070 ceph-mon[39923]: from='client.25792 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: osdmap e129: 8 total, 8 up, 8 in 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/644866528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:19 smithi155 ceph-mon[32688]: from='client.25792 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:20.346 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] Global test environment set-up. 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: 2024-02-10T07:40:18.002+0000 7efd42e38780 -1 failed for service _ceph-mon._tcp 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: 2024-02-10T07:40:18.002+0000 7efd42e38780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (56 ms) 2024-02-10T07:41:20.347 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2010 ms) 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2066 ms total) 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b25ea5720 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2627ad70 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2627cd90 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2627bf20 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262808a0 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2627bd80 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26280d70 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281270 2024-02-10T07:41:20.348 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281790 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281930 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281af0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281c20 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281ee0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262820c0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26282670 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262821d0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262822e0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26282430 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262827a0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26282c80 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26282de0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262833b0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262828d0 2024-02-10T07:41:20.349 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26282a00 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26282f10 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283040 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26282b70 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283190 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283960 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262834c0 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283610 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283720 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283830 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283f90 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283af0 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283c40 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26283d70 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262840c0 2024-02-10T07:41:20.350 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284530 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284660 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284c50 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262841d0 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284320 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262847b0 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262848c0 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262849f0 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284b40 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285220 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284d80 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284e90 2024-02-10T07:41:20.351 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26284fc0 2024-02-10T07:41:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: pgmap v140: 492 pgs: 8 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 126 op/s 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]': finished 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='client.25792 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: osdmap e130: 8 total, 8 up, 8 in 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]: dispatch 2024-02-10T07:41:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]: dispatch 2024-02-10T07:41:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4089259364' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:41:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:41:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:20.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:20.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: pgmap v140: 492 pgs: 8 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 126 op/s 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]': finished 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='client.25792 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: osdmap e130: 8 total, 8 up, 8 in 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]: dispatch 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]: dispatch 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4089259364' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:41:20.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:41:20.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:20 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: pgmap v140: 492 pgs: 8 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 126 op/s 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-13"}]': finished 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='client.25792 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi070-77269-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: osdmap e130: 8 total, 8 up, 8 in 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]: dispatch 2024-02-10T07:41:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]: dispatch 2024-02-10T07:41:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4089259364' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:41:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:41:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:20 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:41:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:41:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]': finished 2024-02-10T07:41:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[29548]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[29548]: osdmap e131: 8 total, 8 up, 8 in 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]': finished 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[39923]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[39923]: osdmap e131: 8 total, 8 up, 8 in 2024-02-10T07:41:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:21 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:41:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-13", "mode": "writeback"}]': finished 2024-02-10T07:41:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:21 smithi155 ceph-mon[32688]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi070-77283-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:21 smithi155 ceph-mon[32688]: osdmap e131: 8 total, 8 up, 8 in 2024-02-10T07:41:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:41:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: pgmap v143: 492 pgs: 8 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 126 op/s 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.0"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.0"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.1"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.1"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.2"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.2"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.3"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.3"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.4"}]: dispatch 2024-02-10T07:41:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.4"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.5"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.5"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.6"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.6"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.7"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.7"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.8"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.8"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.9"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.9"}]: dispatch 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: 162.9 deep-scrub starts 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: 162.9 deep-scrub ok 2024-02-10T07:41:22.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: 162.1 deep-scrub starts 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: 162.1 deep-scrub ok 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: osdmap e132: 8 total, 8 up, 8 in 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2567686256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.25804 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: pgmap v143: 492 pgs: 8 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 126 op/s 2024-02-10T07:41:22.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.0"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.0"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.1"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.1"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.2"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.2"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.3"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.3"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.4"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.4"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.5"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.5"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.6"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.6"}]: dispatch 2024-02-10T07:41:22.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.7"}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.7"}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.8"}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.8"}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.9"}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.9"}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: 162.9 deep-scrub starts 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: 162.9 deep-scrub ok 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: 162.1 deep-scrub starts 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: 162.1 deep-scrub ok 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: osdmap e132: 8 total, 8 up, 8 in 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2567686256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.25804 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:22.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: pgmap v143: 492 pgs: 8 creating+peering, 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 KiB/s wr, 126 op/s 2024-02-10T07:41:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.0"}]: dispatch 2024-02-10T07:41:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.0"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.1"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.1"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.2"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.2"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.3"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.3"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.4"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.4"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.5"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.5"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.6"}]: dispatch 2024-02-10T07:41:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.6"}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.7"}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.7"}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.8"}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.8"}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "162.9"}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "162.9"}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: 162.9 deep-scrub starts 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: 162.9 deep-scrub ok 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: 162.1 deep-scrub starts 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: 162.1 deep-scrub ok 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: osdmap e132: 8 total, 8 up, 8 in 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2567686256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:23.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.25804 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:23.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.3 deep-scrub starts 2024-02-10T07:41:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.3 deep-scrub ok 2024-02-10T07:41:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.4 scrub starts 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.4 scrub ok 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.5 scrub starts 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.5 scrub ok 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.2 scrub starts 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: 162.2 scrub ok 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: from='client.25804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: osdmap e133: 8 total, 8 up, 8 in 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2940091256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: from='client.25810 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.3 deep-scrub starts 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.3 deep-scrub ok 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.4 scrub starts 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.4 scrub ok 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.5 scrub starts 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.5 scrub ok 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.2 scrub starts 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: 162.2 scrub ok 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: from='client.25804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: osdmap e133: 8 total, 8 up, 8 in 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2940091256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: from='client.25810 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:23 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.3 deep-scrub starts 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.3 deep-scrub ok 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.4 scrub starts 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.4 scrub ok 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.5 scrub starts 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.5 scrub ok 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.2 scrub starts 2024-02-10T07:41:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: 162.2 scrub ok 2024-02-10T07:41:24.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: from='client.25804 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi070-77269-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:24.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: osdmap e133: 8 total, 8 up, 8 in 2024-02-10T07:41:24.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2940091256' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:24.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: from='client.25810 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:24.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: 162.6 scrub starts 2024-02-10T07:41:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: 162.6 scrub ok 2024-02-10T07:41:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: pgmap v146: 524 pgs: 8 creating+peering, 64 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: 162.7 deep-scrub starts 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: 162.7 deep-scrub ok 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: from='client.25810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: osdmap e134: 8 total, 8 up, 8 in 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: 162.6 scrub starts 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: 162.6 scrub ok 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: pgmap v146: 524 pgs: 8 creating+peering, 64 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: 162.7 deep-scrub starts 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: 162.7 deep-scrub ok 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: from='client.25810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: osdmap e134: 8 total, 8 up, 8 in 2024-02-10T07:41:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: 162.6 scrub starts 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: 162.6 scrub ok 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: pgmap v146: 524 pgs: 8 creating+peering, 64 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: 162.7 deep-scrub starts 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: 162.7 deep-scrub ok 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: from='client.25810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi070-77283-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: osdmap e134: 8 total, 8 up, 8 in 2024-02-10T07:41:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: 162.0 scrub starts 2024-02-10T07:41:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: 162.0 scrub ok 2024-02-10T07:41:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: 162.8 scrub starts 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: 162.8 scrub ok 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: osdmap e135: 8 total, 8 up, 8 in 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3001037035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: 162.0 scrub starts 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: 162.0 scrub ok 2024-02-10T07:41:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:25.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: 162.8 scrub starts 2024-02-10T07:41:25.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: 162.8 scrub ok 2024-02-10T07:41:25.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: osdmap e135: 8 total, 8 up, 8 in 2024-02-10T07:41:25.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3001037035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:25.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:25.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: 162.0 scrub starts 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: 162.0 scrub ok 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: 162.8 scrub starts 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: 162.8 scrub ok 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: osdmap e135: 8 total, 8 up, 8 in 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3001037035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: pgmap v149: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 6.0 KiB/s wr, 77 op/s 2024-02-10T07:41:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: osdmap e136: 8 total, 8 up, 8 in 2024-02-10T07:41:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3727048356' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: from='client.35819 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: from='client.35819 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[29548]: osdmap e137: 8 total, 8 up, 8 in 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: pgmap v149: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 6.0 KiB/s wr, 77 op/s 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: osdmap e136: 8 total, 8 up, 8 in 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3727048356' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: from='client.35819 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: from='client.35819 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:27 smithi070 ceph-mon[39923]: osdmap e137: 8 total, 8 up, 8 in 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: pgmap v149: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 6.0 KiB/s wr, 77 op/s 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi070-77269-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: osdmap e136: 8 total, 8 up, 8 in 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3727048356' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: from='client.35819 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: from='client.35819 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi070-77283-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:27 smithi155 ceph-mon[32688]: osdmap e137: 8 total, 8 up, 8 in 2024-02-10T07:41:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[39923]: osdmap e138: 8 total, 8 up, 8 in 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[29548]: osdmap e138: 8 total, 8 up, 8 in 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:28 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-mon[32688]: osdmap e138: 8 total, 8 up, 8 in 2024-02-10T07:41:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/914333239' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]: dispatch 2024-02-10T07:41:29.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:41:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started api_snapshots: Running main() from gmock_main.cc 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (5154 ms) 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (5026 ms) 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2776 ms) 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-10T07:41:29.310 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2556 ms) 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (15512 ms total) 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4327 ms) 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4091 ms) 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4639 ms) 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13057 ms total) 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-10T07:41:29.311 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3019 ms) 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1983 ms) 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2003 ms) 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1997 ms) 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9002 ms total) 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4025 ms) 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4014 ms) 2024-02-10T07:41:29.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8040 ms total) 2024-02-10T07:41:29.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: 2024-02-10T07:41:29.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-10T07:41:29.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (71342 ms total) 2024-02-10T07:41:29.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-02-10T07:41:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: pgmap v152: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-02-10T07:41:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3643323186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: from='client.25828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: from='client.25828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: osdmap e139: 8 total, 8 up, 8 in 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/57242814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: pgmap v152: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3643323186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: from='client.25828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: from='client.25828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: osdmap e139: 8 total, 8 up, 8 in 2024-02-10T07:41:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/57242814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: pgmap v152: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 57 KiB/s rd, 76 op/s 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3643323186' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: from='client.25828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi070-77602-15"}]': finished 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: from='client.25828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi070-77269-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: osdmap e139: 8 total, 8 up, 8 in 2024-02-10T07:41:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/57242814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:30.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:30.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:30.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/57242814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[29548]: osdmap e140: 8 total, 8 up, 8 in 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1968348002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[29548]: from='client.25840 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/57242814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[39923]: osdmap e140: 8 total, 8 up, 8 in 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1968348002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:30.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:30 smithi070 ceph-mon[39923]: from='client.25840 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/57242814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:30 smithi155 ceph-mon[32688]: osdmap e140: 8 total, 8 up, 8 in 2024-02-10T07:41:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1968348002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:30 smithi155 ceph-mon[32688]: from='client.25840 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: pgmap v155: 484 pgs: 32 unknown, 32 creating+peering, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.25840 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: osdmap e141: 8 total, 8 up, 8 in 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4140938435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2"}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.35843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: pgmap v155: 484 pgs: 32 unknown, 32 creating+peering, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.25840 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: osdmap e141: 8 total, 8 up, 8 in 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4140938435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2"}]: dispatch 2024-02-10T07:41:31.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.35843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:31 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: pgmap v155: 484 pgs: 32 unknown, 32 creating+peering, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T07:41:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.25840 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77617-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: osdmap e141: 8 total, 8 up, 8 in 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4140938435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2"}]: dispatch 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.35843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:31.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:31 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:32.341 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:41:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:41:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.35843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: osdmap e142: 8 total, 8 up, 8 in 2024-02-10T07:41:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/696927734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.25855 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.35843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: osdmap e142: 8 total, 8 up, 8 in 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/696927734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.25855 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:32.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:32 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.35843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77269-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi070-77479-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: osdmap e142: 8 total, 8 up, 8 in 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/696927734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.25855 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:32.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:32 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T07:41:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[29548]: pgmap v158: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[29548]: from='client.25855 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T07:41:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[29548]: osdmap e143: 8 total, 8 up, 8 in 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[39923]: pgmap v158: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[39923]: from='client.25855 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[39923]: osdmap e143: 8 total, 8 up, 8 in 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T07:41:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:33 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T07:41:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:33 smithi155 ceph-mon[32688]: pgmap v158: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:33 smithi155 ceph-mon[32688]: from='client.25855 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi070-77283-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:33 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T07:41:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:33 smithi155 ceph-mon[32688]: osdmap e143: 8 total, 8 up, 8 in 2024-02-10T07:41:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T07:41:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:33 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T07:41:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T07:41:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[39923]: osdmap e144: 8 total, 8 up, 8 in 2024-02-10T07:41:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T07:41:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/386500496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[39923]: from='client.35870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[29548]: osdmap e144: 8 total, 8 up, 8 in 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/386500496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:34 smithi070 ceph-mon[29548]: from='client.35870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:34 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T07:41:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:34 smithi155 ceph-mon[32688]: osdmap e144: 8 total, 8 up, 8 in 2024-02-10T07:41:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1987198651' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T07:41:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:34 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T07:41:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/386500496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:34 smithi155 ceph-mon[32688]: from='client.35870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: pgmap v161: 484 pgs: 52 unknown, 432 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T07:41:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]': finished 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.35870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: osdmap e145: 8 total, 8 up, 8 in 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4282829255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi070-77283-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: pgmap v161: 484 pgs: 52 unknown, 432 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]': finished 2024-02-10T07:41:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.35870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: osdmap e145: 8 total, 8 up, 8 in 2024-02-10T07:41:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4282829255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi070-77283-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:35 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: pgmap v161: 484 pgs: 52 unknown, 432 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.34715 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi070-77479-1","app":"app1","key":"key1"}]': finished 2024-02-10T07:41:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.35870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi070-77269-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: osdmap e145: 8 total, 8 up, 8 in 2024-02-10T07:41:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4282829255' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi070-77283-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:35 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: 2024-02-10T07:41:36.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: seed 77479 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (396 ms) 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (14 ms) 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (7 ms) 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (12 ms) 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (5 ms) 2024-02-10T07:41:36.317 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (5 ms) 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (1 ms) 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (13 ms) 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-10T07:41:36.318 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1479 ms) 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (459 ms) 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: done waiting, doing copies 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: done waiting 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60620 ms) 2024-02-10T07:41:36.319 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4600 ms) 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-10T07:41:36.320 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-10T07:41:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4282829255' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi070-77283-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4282829255' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi070-77283-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4282829255' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi070-77283-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:37.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: pgmap v164: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:41:37.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: osdmap e146: 8 total, 8 up, 8 in 2024-02-10T07:41:37.897 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: osdmap e147: 8 total, 8 up, 8 in 2024-02-10T07:41:37.898 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.899 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4081516339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:37.899 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.899 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:37 smithi155 ceph-mon[32688]: from='client.35903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:37.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: pgmap v164: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:41:37.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: osdmap e146: 8 total, 8 up, 8 in 2024-02-10T07:41:37.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.996 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.997 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:37.997 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:37.997 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:37.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:37.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:37.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: osdmap e147: 8 total, 8 up, 8 in 2024-02-10T07:41:37.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4081516339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:37.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.998 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[29548]: from='client.35903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:37.999 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: pgmap v164: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:41:37.999 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:37.999 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: osdmap e146: 8 total, 8 up, 8 in 2024-02-10T07:41:38.000 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:38.000 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:38.000 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:38.000 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:38.001 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:38.001 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:38.001 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi070-77617-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:38.001 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77479-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:38.001 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: osdmap e147: 8 total, 8 up, 8 in 2024-02-10T07:41:38.001 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:38.002 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4081516339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:38.002 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:38.002 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:37 smithi070 ceph-mon[39923]: from='client.35903 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:38.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:38.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:38 smithi155 ceph-mon[32688]: pgmap v167: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:38.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:38 smithi155 ceph-mon[32688]: from='client.35903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:38.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:38 smithi155 ceph-mon[32688]: osdmap e148: 8 total, 8 up, 8 in 2024-02-10T07:41:38.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3337907362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi070-77283-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[39923]: pgmap v167: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[39923]: from='client.35903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[39923]: osdmap e148: 8 total, 8 up, 8 in 2024-02-10T07:41:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3337907362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi070-77283-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[29548]: pgmap v167: 428 pgs: 40 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-10T07:41:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[29548]: from='client.35903 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi070-77269-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[29548]: osdmap e148: 8 total, 8 up, 8 in 2024-02-10T07:41:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3337907362' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi070-77283-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:39.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:41:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:41:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3337907362' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi070-77283-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[29548]: osdmap e149: 8 total, 8 up, 8 in 2024-02-10T07:41:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:40.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3337907362' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi070-77283-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[39923]: osdmap e149: 8 total, 8 up, 8 in 2024-02-10T07:41:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:39 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77479-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3337907362' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi070-77283-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:39 smithi155 ceph-mon[32688]: osdmap e149: 8 total, 8 up, 8 in 2024-02-10T07:41:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:41.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: pgmap v170: 436 pgs: 32 creating+peering, 8 unknown, 396 active+clean; 592 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:41:41.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:41.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: osdmap e150: 8 total, 8 up, 8 in 2024-02-10T07:41:41.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4113222781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: from='client.35915 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: pgmap v170: 436 pgs: 32 creating+peering, 8 unknown, 396 active+clean; 592 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: osdmap e150: 8 total, 8 up, 8 in 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4113222781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: from='client.35915 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: pgmap v170: 436 pgs: 32 creating+peering, 8 unknown, 396 active+clean; 592 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: osdmap e150: 8 total, 8 up, 8 in 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4113222781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: from='client.35915 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:42.328 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:41:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: seed 77617 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (5160 ms) 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (5020 ms) 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-10T07:41:42.335 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2778 ms) 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2554 ms) 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (5365 ms) 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (20877 ms total) 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4152 ms) 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3798 ms) 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6013 ms) 2024-02-10T07:41:42.336 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3993 ms) 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2069 ms) 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1361-g290212a1/rpm/el9/BUILD/ceph-19.0.0-1361-g290212a1/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi070-77617-7 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17934 ms) 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37960 ms total) 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-10T07:41:42.337 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5045 ms) 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5054 ms total) 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3009 ms) 2024-02-10T07:41:42.338 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-10T07:41:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: from='client.35915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: osdmap e151: 8 total, 8 up, 8 in 2024-02-10T07:41:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2728139671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi070-77283-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: from='client.35915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: osdmap e151: 8 total, 8 up, 8 in 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2728139671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi070-77283-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:42 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: from='client.35915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi070-77269-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: osdmap e151: 8 total, 8 up, 8 in 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2728139671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi070-77283-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:42 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[39923]: pgmap v173: 500 pgs: 104 unknown, 396 active+clean; 592 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:41:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2728139671' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi070-77283-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[39923]: osdmap e152: 8 total, 8 up, 8 in 2024-02-10T07:41:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:43.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[29548]: pgmap v173: 500 pgs: 104 unknown, 396 active+clean; 592 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:41:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2728139671' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi070-77283-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[29548]: osdmap e152: 8 total, 8 up, 8 in 2024-02-10T07:41:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:43 smithi155 ceph-mon[32688]: pgmap v173: 500 pgs: 104 unknown, 396 active+clean; 592 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:41:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:43 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77479-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2728139671' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi070-77283-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:43 smithi155 ceph-mon[32688]: osdmap e152: 8 total, 8 up, 8 in 2024-02-10T07:41:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:44 smithi070 ceph-mon[29548]: osdmap e153: 8 total, 8 up, 8 in 2024-02-10T07:41:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1677637113' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi070-77269-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:44.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:44 smithi070 ceph-mon[39923]: osdmap e153: 8 total, 8 up, 8 in 2024-02-10T07:41:44.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1677637113' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi070-77269-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:44.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:44 smithi155 ceph-mon[32688]: osdmap e153: 8 total, 8 up, 8 in 2024-02-10T07:41:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1677637113' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi070-77269-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: pgmap v176: 436 pgs: 21 creating+peering, 15 unknown, 400 active+clean; 592 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1677637113' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi070-77269-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: osdmap e154: 8 total, 8 up, 8 in 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/244009399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: from='client.35933 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: pgmap v176: 436 pgs: 21 creating+peering, 15 unknown, 400 active+clean; 592 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1677637113' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi070-77269-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: osdmap e154: 8 total, 8 up, 8 in 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/244009399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: from='client.35933 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: pgmap v176: 436 pgs: 21 creating+peering, 15 unknown, 400 active+clean; 592 KiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1677637113' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi070-77269-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: osdmap e154: 8 total, 8 up, 8 in 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/244009399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: from='client.35933 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.35933 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: osdmap e155: 8 total, 8 up, 8 in 2024-02-10T07:41:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: osdmap e156: 8 total, 8 up, 8 in 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3819016610' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[29548]: from='client.35939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.35933 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: osdmap e155: 8 total, 8 up, 8 in 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: osdmap e156: 8 total, 8 up, 8 in 2024-02-10T07:41:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3819016610' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:46.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:46 smithi070 ceph-mon[39923]: from='client.35939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.35933 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi070-77283-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: osdmap e155: 8 total, 8 up, 8 in 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1682710634' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]: dispatch 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.35897 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77479-24"}]': finished 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: osdmap e156: 8 total, 8 up, 8 in 2024-02-10T07:41:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3819016610' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:46 smithi155 ceph-mon[32688]: from='client.35939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: pgmap v179: 428 pgs: 32 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:41:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.35939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: osdmap e157: 8 total, 8 up, 8 in 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2934859423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/715267345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.25936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[29548]: from='client.25933 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: pgmap v179: 428 pgs: 32 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.35939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: osdmap e157: 8 total, 8 up, 8 in 2024-02-10T07:41:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2934859423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/715267345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.25936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:47 smithi070 ceph-mon[39923]: from='client.25933 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: pgmap v179: 428 pgs: 32 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:41:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.35939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi070-77269-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: osdmap e157: 8 total, 8 up, 8 in 2024-02-10T07:41:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2934859423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/715267345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.25936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:47 smithi155 ceph-mon[32688]: from='client.25933 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:48.950 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:48.950 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:48.950 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-mon[32688]: from='client.25936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:48.950 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-mon[32688]: from='client.25933 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:48.950 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-mon[32688]: osdmap e158: 8 total, 8 up, 8 in 2024-02-10T07:41:48.950 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:48.950 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[29548]: from='client.25936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[29548]: from='client.25933 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[29548]: osdmap e158: 8 total, 8 up, 8 in 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[39923]: from='client.25936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi070-77283-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[39923]: from='client.25933 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi070-77479-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[39923]: osdmap e158: 8 total, 8 up, 8 in 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3821584165' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:48 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]: dispatch 2024-02-10T07:41:49.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:41:48 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:41:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: pgmap v182: 484 pgs: 96 unknown, 388 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:41:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: osdmap e159: 8 total, 8 up, 8 in 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3956060871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi070-77269-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: pgmap v182: 484 pgs: 96 unknown, 388 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:41:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: osdmap e159: 8 total, 8 up, 8 in 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3956060871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi070-77269-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: pgmap v182: 484 pgs: 96 unknown, 388 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:41:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.35891 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi070-77617-16"}]': finished 2024-02-10T07:41:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: osdmap e159: 8 total, 8 up, 8 in 2024-02-10T07:41:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3956060871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi070-77269-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:50.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: pgmap v185: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:41:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3956060871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi070-77269-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: osdmap e160: 8 total, 8 up, 8 in 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1457414731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2538282985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.35963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.25948 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: pgmap v185: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3956060871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi070-77269-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: osdmap e160: 8 total, 8 up, 8 in 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1457414731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2538282985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.35963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.25948 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: pgmap v185: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3956060871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi070-77269-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: osdmap e160: 8 total, 8 up, 8 in 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1457414731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:41:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2538282985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.35963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.25948 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:51.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:52.309 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:41:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:41:52.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.35963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:52.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.25948 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:52.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: osdmap e161: 8 total, 8 up, 8 in 2024-02-10T07:41:52.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.35963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.25948 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: osdmap e161: 8 total, 8 up, 8 in 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:52.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:52.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.35963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi070-77479-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.25948 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi070-77283-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: osdmap e161: 8 total, 8 up, 8 in 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:52.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:52.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:53.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: pgmap v188: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:41:53.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:53.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:41:53.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:53.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: osdmap e162: 8 total, 8 up, 8 in 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: pgmap v188: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: osdmap e162: 8 total, 8 up, 8 in 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: pgmap v188: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi070-77269-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: osdmap e162: 8 total, 8 up, 8 in 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[39923]: osdmap e163: 8 total, 8 up, 8 in 2024-02-10T07:41:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2650037993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1365426992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi070-77479-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[39923]: from='client.35984 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[29548]: osdmap e163: 8 total, 8 up, 8 in 2024-02-10T07:41:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2650037993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1365426992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi070-77479-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[29548]: from='client.35984 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:54 smithi155 ceph-mon[32688]: osdmap e163: 8 total, 8 up, 8 in 2024-02-10T07:41:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2650037993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1365426992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi070-77479-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:54 smithi155 ceph-mon[32688]: from='client.35984 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[29548]: pgmap v191: 460 pgs: 59 creating+peering, 5 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:41:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1365426992' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi070-77479-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[29548]: from='client.35984 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[29548]: osdmap e164: 8 total, 8 up, 8 in 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[39923]: pgmap v191: 460 pgs: 59 creating+peering, 5 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1365426992' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi070-77479-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[39923]: from='client.35984 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[39923]: osdmap e164: 8 total, 8 up, 8 in 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:55 smithi155 ceph-mon[32688]: pgmap v191: 460 pgs: 59 creating+peering, 5 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:41:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:55 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi070-77269-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1365426992' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi070-77479-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:55 smithi155 ceph-mon[32688]: from='client.35984 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi070-77283-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:55 smithi155 ceph-mon[32688]: osdmap e164: 8 total, 8 up, 8 in 2024-02-10T07:41:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:41:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: osdmap e165: 8 total, 8 up, 8 in 2024-02-10T07:41:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:56.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: osdmap e166: 8 total, 8 up, 8 in 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3453270658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi070-77283-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: osdmap e165: 8 total, 8 up, 8 in 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:56.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: osdmap e166: 8 total, 8 up, 8 in 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3453270658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi070-77283-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:56.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:56 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: osdmap e165: 8 total, 8 up, 8 in 2024-02-10T07:41:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: osdmap e166: 8 total, 8 up, 8 in 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3453270658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi070-77283-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:41:56.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:56 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[39923]: pgmap v194: 404 pgs: 8 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:41:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]': finished 2024-02-10T07:41:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[29548]: pgmap v194: 404 pgs: 8 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:41:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]': finished 2024-02-10T07:41:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:57 smithi155 ceph-mon[32688]: pgmap v194: 404 pgs: 8 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:41:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:57 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:41:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-13"}]': finished 2024-02-10T07:41:58.302 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosS api_aio: Running main() from gmock_main.cc 2024-02-10T07:41:58.302 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-02-10T07:41:58.302 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [----------] Global test environment set-up. 2024-02-10T07:41:58.302 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-02-10T07:41:58.302 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-10T07:41:58.302 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.TooBig (10469 ms) 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (6755 ms) 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (5554 ms) 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3557 ms) 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (4362 ms) 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2786 ms) 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3086 ms) 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-10T07:41:58.303 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2853 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3007 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3003 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3019 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2987 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2992 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3017 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.Flush (2984 ms) 2024-02-10T07:41:58.304 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3007 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3000 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2996 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3015 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3004 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3258 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (2755 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3016 ms) 2024-02-10T07:41:58.305 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3013 ms) 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2948 ms) 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3257 ms) 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [----------] 26 tests from LibRadosAio (93700 ms total) 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-10T07:41:58.306 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-10T07:41:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3453270658' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi070-77283-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: osdmap e167: 8 total, 8 up, 8 in 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[29548]: osdmap e168: 8 total, 8 up, 8 in 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3453270658' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi070-77283-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: osdmap e167: 8 total, 8 up, 8 in 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:41:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:59.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:58 smithi070 ceph-mon[39923]: osdmap e168: 8 total, 8 up, 8 in 2024-02-10T07:41:59.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3453270658' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi070-77283-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2315194730' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: osdmap e167: 8 total, 8 up, 8 in 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]: dispatch 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi070-77479-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi070-77269-27"}]': finished 2024-02-10T07:41:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:58 smithi155 ceph-mon[32688]: osdmap e168: 8 total, 8 up, 8 in 2024-02-10T07:42:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: pgmap v197: 428 pgs: 32 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: osdmap e169: 8 total, 8 up, 8 in 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4109499569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.36005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: pgmap v197: 428 pgs: 32 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: osdmap e169: 8 total, 8 up, 8 in 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4109499569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.36005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:41:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: pgmap v197: 428 pgs: 32 unknown, 396 active+clean; 597 KiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi070-77269-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: osdmap e169: 8 total, 8 up, 8 in 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4109499569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.36005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:00.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:41:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[29548]: pgmap v200: 436 pgs: 64 unknown, 8 creating+peering, 364 active+clean; 609 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T07:42:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[29548]: from='client.36005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[29548]: osdmap e170: 8 total, 8 up, 8 in 2024-02-10T07:42:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[39923]: pgmap v200: 436 pgs: 64 unknown, 8 creating+peering, 364 active+clean; 609 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T07:42:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[39923]: from='client.36005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[39923]: osdmap e170: 8 total, 8 up, 8 in 2024-02-10T07:42:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:01 smithi155 ceph-mon[32688]: pgmap v200: 436 pgs: 64 unknown, 8 creating+peering, 364 active+clean; 609 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T07:42:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:01 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:01 smithi155 ceph-mon[32688]: from='client.36005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi070-77283-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:01 smithi155 ceph-mon[32688]: osdmap e170: 8 total, 8 up, 8 in 2024-02-10T07:42:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:01 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:02.312 INFO:tasks.workunit.client.0.smithi070.stdout:(67637 ms total) 2024-02-10T07:42:02.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: 2024-02-10T07:42:02.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-10T07:42:02.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-10T07:42:02.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (120 ms) 2024-02-10T07:42:02.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (120 ms total) 2024-02-10T07:42:02.312 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (48 ms) 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (6 ms) 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (54 ms total) 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (273 ms) 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (12 ms) 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (285 ms total) 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: 2024-02-10T07:42:02.313 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (44 ms) 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (8 ms) 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (52 ms total) 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1025 ms) 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1025 ms total) 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (104369 ms total) 2024-02-10T07:42:02.314 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-10T07:42:02.318 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:42:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:42:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:42:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: osdmap e171: 8 total, 8 up, 8 in 2024-02-10T07:42:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: osdmap e172: 8 total, 8 up, 8 in 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: osdmap e171: 8 total, 8 up, 8 in 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:02.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:02.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:02.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:42:02.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:02.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: osdmap e172: 8 total, 8 up, 8 in 2024-02-10T07:42:02.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi070-77269-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: osdmap e171: 8 total, 8 up, 8 in 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1547935336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]: dispatch 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.35993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi070-77479-36"}]': finished 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: osdmap e172: 8 total, 8 up, 8 in 2024-02-10T07:42:02.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: pgmap v203: 404 pgs: 40 unknown, 364 active+clean; 609 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T07:42:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1347783810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.36011 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]': finished 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.36011 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: osdmap e173: 8 total, 8 up, 8 in 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: pgmap v203: 404 pgs: 40 unknown, 364 active+clean; 609 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1347783810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.36011 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]': finished 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.36011 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: osdmap e173: 8 total, 8 up, 8 in 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]: dispatch 2024-02-10T07:42:03.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]: dispatch 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: pgmap v203: 404 pgs: 40 unknown, 364 active+clean; 609 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1347783810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.36011 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-15"}]': finished 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.36011 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi070-77283-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: osdmap e173: 8 total, 8 up, 8 in 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]: dispatch 2024-02-10T07:42:03.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]: dispatch 2024-02-10T07:42:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]': finished 2024-02-10T07:42:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[29548]: osdmap e174: 8 total, 8 up, 8 in 2024-02-10T07:42:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]': finished 2024-02-10T07:42:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[39923]: osdmap e174: 8 total, 8 up, 8 in 2024-02-10T07:42:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:04 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:04 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:04 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:04 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-15", "mode": "writeback"}]': finished 2024-02-10T07:42:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:04 smithi155 ceph-mon[32688]: osdmap e174: 8 total, 8 up, 8 in 2024-02-10T07:42:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/267192338' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:04 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]: dispatch 2024-02-10T07:42:05.328 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: pgmap v206: 428 pgs: 12 unknown, 416 active+clean; 609 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-02-10T07:42:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: pgmap v206: 428 pgs: 12 unknown, 416 active+clean; 609 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-02-10T07:42:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: osdmap e175: 8 total, 8 up, 8 in 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2297847032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi070-77283-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: pgmap v206: 428 pgs: 12 unknown, 416 active+clean; 609 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.5 KiB/s wr, 3 op/s 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: osdmap e175: 8 total, 8 up, 8 in 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2297847032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi070-77283-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:05.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:05 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: from='client.36002 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi070-77269-28"}]': finished 2024-02-10T07:42:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: osdmap e175: 8 total, 8 up, 8 in 2024-02-10T07:42:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2297847032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi070-77283-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:05 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:06.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi070-77269-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:06.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:06.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:06.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:06.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2297847032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi070-77283-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi070-77269-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: osdmap e176: 8 total, 8 up, 8 in 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi070-77269-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:06.659 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi070-77269-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2297847032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi070-77283-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi070-77269-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: osdmap e176: 8 total, 8 up, 8 in 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi070-77269-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:06.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi070-77269-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2297847032' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi070-77283-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi070-77269-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: osdmap e176: 8 total, 8 up, 8 in 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3138291668' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi070-77269-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:06.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]: dispatch 2024-02-10T07:42:06.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]: dispatch 2024-02-10T07:42:07.314 INFO:tasks.workunit.client.0.smithi070.stdout:napshotsECPP.SnapRemovePP (2010 ms) 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2011 ms) 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2006 ms) 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9037 ms total) 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4034 ms) 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3953 ms) 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-10T07:42:07.315 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4000 ms) 2024-02-10T07:42:07.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11987 ms total) 2024-02-10T07:42:07.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: 2024-02-10T07:42:07.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-10T07:42:07.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (109338 ms total) 2024-02-10T07:42:07.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-10T07:42:07.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-02-10T07:42:07.316 INFO:tasks.workunit.client.0.smithi070.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-10T07:42:07.555 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-10T07:42:07.555 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-02-10T07:42:07.555 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-10T07:42:07.555 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-02-10T07:42:07.555 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-10T07:42:07.555 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (10837 ms) 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (20605 ms) 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6123 ms) 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2863 ms) 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3010 ms) 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2992 ms) 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3000 ms) 2024-02-10T07:42:07.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3001 ms) 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2999 ms) 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3014 ms) 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2989 ms) 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3001 ms) 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3011 ms) 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-10T07:42:07.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (2998 ms) 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3015 ms) 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2999 ms) 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3008 ms) 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3007 ms) 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3012 ms) 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3015 ms) 2024-02-10T07:42:07.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2944 ms) 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3023 ms) 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3012 ms) 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2969 ms) 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3008 ms) 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2994 ms) 2024-02-10T07:42:07.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-10T07:42:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[29548]: pgmap v209: 420 pgs: 32 unknown, 388 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[29548]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:42:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[39923]: pgmap v209: 420 pgs: 32 unknown, 388 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:07 smithi070 ceph-mon[39923]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:42:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:07 smithi155 ceph-mon[32688]: pgmap v209: 420 pgs: 32 unknown, 388 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:07 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:07 smithi155 ceph-mon[32688]: from='client.25951 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi070-77617-21"}]': finished 2024-02-10T07:42:08.963 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]': finished 2024-02-10T07:42:08.963 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:08 smithi155 ceph-mon[32688]: osdmap e177: 8 total, 8 up, 8 in 2024-02-10T07:42:08.963 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:08.963 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi070-77269-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:08.963 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:08 smithi155 ceph-mon[32688]: osdmap e178: 8 total, 8 up, 8 in 2024-02-10T07:42:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]': finished 2024-02-10T07:42:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[39923]: osdmap e177: 8 total, 8 up, 8 in 2024-02-10T07:42:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi070-77269-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[39923]: osdmap e178: 8 total, 8 up, 8 in 2024-02-10T07:42:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-15"}]': finished 2024-02-10T07:42:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[29548]: osdmap e177: 8 total, 8 up, 8 in 2024-02-10T07:42:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi070-77269-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:08 smithi070 ceph-mon[29548]: osdmap e178: 8 total, 8 up, 8 in 2024-02-10T07:42:09.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:42:08 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:42:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: pgmap v212: 388 pgs: 388 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1372264465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.36026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.36026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]': finished 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: osdmap e179: 8 total, 8 up, 8 in 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: pgmap v212: 388 pgs: 388 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1372264465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.36026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.36026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]': finished 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: osdmap e179: 8 total, 8 up, 8 in 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:09.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:09.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:09.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:09.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:09 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: pgmap v212: 388 pgs: 388 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1372264465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.36026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.36026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi070-77283-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]': finished 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: osdmap e179: 8 total, 8 up, 8 in 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:09 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: pgmap v215: 428 pgs: 32 unknown, 40 creating+peering, 356 active+clean; 585 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: osdmap e180: 8 total, 8 up, 8 in 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:11 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:11.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: pgmap v215: 428 pgs: 32 unknown, 40 creating+peering, 356 active+clean; 585 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:11.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:11.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: osdmap e180: 8 total, 8 up, 8 in 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: pgmap v215: 428 pgs: 32 unknown, 40 creating+peering, 356 active+clean; 585 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: osdmap e180: 8 total, 8 up, 8 in 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:11.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:11 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T07:42:12.329 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:42:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:42:12.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:12.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]': finished 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: osdmap e181: 8 total, 8 up, 8 in 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3755148240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.26011 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]': finished 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: from='client.26011 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[29548]: osdmap e182: 8 total, 8 up, 8 in 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]': finished 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: osdmap e181: 8 total, 8 up, 8 in 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3755148240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.26011 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]': finished 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: from='client.26011 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:12.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:12 smithi070 ceph-mon[39923]: osdmap e182: 8 total, 8 up, 8 in 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pg_num","val":"11"}]': finished 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: osdmap e181: 8 total, 8 up, 8 in 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3755148240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]: dispatch 2024-02-10T07:42:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:12.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.26011 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:12.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/819583147' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi070-77269-29"}]': finished 2024-02-10T07:42:12.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-17"}]': finished 2024-02-10T07:42:12.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: from='client.26011 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi070-77283-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:12.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:12 smithi155 ceph-mon[32688]: osdmap e182: 8 total, 8 up, 8 in 2024-02-10T07:42:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: pgmap v218: 420 pgs: 64 unknown, 356 active+clean; 585 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]': finished 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: osdmap e183: 8 total, 8 up, 8 in 2024-02-10T07:42:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: pgmap v218: 420 pgs: 64 unknown, 356 active+clean; 585 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:13.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]': finished 2024-02-10T07:42:13.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:13.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: osdmap e183: 8 total, 8 up, 8 in 2024-02-10T07:42:13.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: pgmap v218: 420 pgs: 64 unknown, 356 active+clean; 585 KiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-17", "mode": "writeback"}]': finished 2024-02-10T07:42:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi070-77269-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: osdmap e183: 8 total, 8 up, 8 in 2024-02-10T07:42:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:13.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi070-77348-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T07:42:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi070-77348-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T07:42:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi070-77348-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T07:42:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:14 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi070-77348-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T07:42:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:14 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:14 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi070-77348-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T07:42:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:14 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:14 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi070-77348-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T07:42:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: pgmap v221: 388 pgs: 388 active+clean; 585 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:42:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: osdmap e184: 8 total, 8 up, 8 in 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3253186428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi070-77283-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]': finished 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3253186428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi070-77283-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: osdmap e185: 8 total, 8 up, 8 in 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: pgmap v221: 388 pgs: 388 active+clean; 585 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: osdmap e184: 8 total, 8 up, 8 in 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3253186428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi070-77283-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]': finished 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3253186428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi070-77283-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: osdmap e185: 8 total, 8 up, 8 in 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:15.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:15.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:15 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: pgmap v221: 388 pgs: 388 active+clean; 585 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:42:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: osdmap e184: 8 total, 8 up, 8 in 2024-02-10T07:42:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]: dispatch 2024-02-10T07:42:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3253186428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi070-77283-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi070-77269-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-17"}]': finished 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3253186428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi070-77283-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: osdmap e185: 8 total, 8 up, 8 in 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:16.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:15 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T07:42:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[29548]: pgmap v224: 428 pgs: 40 unknown, 388 active+clean; 585 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:42:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[29548]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T07:42:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[29548]: osdmap e186: 8 total, 8 up, 8 in 2024-02-10T07:42:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[39923]: pgmap v224: 428 pgs: 40 unknown, 388 active+clean; 585 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:42:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[39923]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T07:42:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[39923]: osdmap e186: 8 total, 8 up, 8 in 2024-02-10T07:42:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:17 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:17 smithi155 ceph-mon[32688]: pgmap v224: 428 pgs: 40 unknown, 388 active+clean; 585 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:42:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:17 smithi155 ceph-mon[32688]: from='client.24700 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi070-77348-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T07:42:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:17 smithi155 ceph-mon[32688]: osdmap e186: 8 total, 8 up, 8 in 2024-02-10T07:42:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4132713586' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T07:42:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:17 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:18.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: osdmap e187: 8 total, 8 up, 8 in 2024-02-10T07:42:18.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3951188733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:18.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.36047 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: osdmap e187: 8 total, 8 up, 8 in 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3951188733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.36047 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:18.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:18 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: osdmap e187: 8 total, 8 up, 8 in 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3951188733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.36047 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:18.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:18.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:19.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:42:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:42:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: pgmap v227: 388 pgs: 64 unknown, 324 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.36047 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: osdmap e188: 8 total, 8 up, 8 in 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3951188733' entity='client.admin' cmd=[{ 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "prefix": "osd pool set", 2024-02-10T07:42:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "var": "eio", 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "val": "true" 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: }]: dispatch 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: from='client.36047 ' entity='client.admin' cmd=[{ 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "prefix": "osd pool set", 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "var": "eio", 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: "val": "true" 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[29548]: }]: dispatch 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: pgmap v227: 388 pgs: 64 unknown, 324 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.36047 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:19.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: osdmap e188: 8 total, 8 up, 8 in 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3951188733' entity='client.admin' cmd=[{ 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "prefix": "osd pool set", 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "var": "eio", 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "val": "true" 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: }]: dispatch 2024-02-10T07:42:19.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: from='client.36047 ' entity='client.admin' cmd=[{ 2024-02-10T07:42:19.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "prefix": "osd pool set", 2024-02-10T07:42:19.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:19.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "var": "eio", 2024-02-10T07:42:19.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: "val": "true" 2024-02-10T07:42:19.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:19 smithi070 ceph-mon[39923]: }]: dispatch 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: pgmap v227: 388 pgs: 64 unknown, 324 active+clean; 583 KiB data, 581 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.36047 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi070-77283-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi070-77348-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: osdmap e188: 8 total, 8 up, 8 in 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3344528176' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]: dispatch 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3951188733' entity='client.admin' cmd=[{ 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "prefix": "osd pool set", 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "var": "eio", 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "val": "true" 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: }]: dispatch 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: from='client.36047 ' entity='client.admin' cmd=[{ 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "prefix": "osd pool set", 2024-02-10T07:42:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:19.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "var": "eio", 2024-02-10T07:42:19.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: "val": "true" 2024-02-10T07:42:19.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:19 smithi155 ceph-mon[32688]: }]: dispatch 2024-02-10T07:42:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.36047 ' entity='client.admin' cmd='[{ 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: "prefix": "osd pool set", 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: "var": "eio", 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: "val": "true" 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: }]': finished 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: osdmap e189: 8 total, 8 up, 8 in 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:20.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.36047 ' entity='client.admin' cmd='[{ 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: "prefix": "osd pool set", 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: "var": "eio", 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: "val": "true" 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: }]': finished 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: osdmap e189: 8 total, 8 up, 8 in 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:20.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:20.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:20 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.36035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi070-77269-30"}]': finished 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.36047 ' entity='client.admin' cmd='[{ 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: "prefix": "osd pool set", 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: "pool": "PoolEIOFlag_smithi070-77283-33", 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: "var": "eio", 2024-02-10T07:42:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: "val": "true" 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: }]': finished 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: osdmap e189: 8 total, 8 up, 8 in 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:20.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:20 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:21.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAintry=3 expected=3 2024-02-10T07:42:21.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:c4fdafeb:::6:head 2024-02-10T07:42:21.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:c4fdafeb:::6:head expected=16:c4fdafeb:::6:head 2024-02-10T07:42:21.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:c4fdafeb:::6:head -> 6 2024-02-10T07:42:21.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=6 expected=6 2024-02-10T07:42:21.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:bd63b0f1:::8:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:bd63b0f1:::8:head expected=16:bd63b0f1:::8:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:bd63b0f1:::8:head -> 8 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=8 expected=8 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:863748b0:::15:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:863748b0:::15:head expected=16:863748b0:::15:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:863748b0:::15:head -> 15 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=15 expected=15 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:6cac518f:::0:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:6cac518f:::0:head expected=16:6cac518f:::0:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:6cac518f:::0:head -> 0 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=0 expected=0 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:62a1935d:::14:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:62a1935d:::14:head expected=16:62a1935d:::14:head 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:62a1935d:::14:head -> 14 2024-02-10T07:42:21.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=14 expected=14 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:02547ec2:::1:head 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:02547ec2:::1:head expected=16:02547ec2:::1:head 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:02547ec2:::1:head -> 1 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=1 expected=1 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:52ea6a34:::10:head 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:52ea6a34:::10:head expected=16:52ea6a34:::10:head 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:52ea6a34:::10:head -> 10 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=10 expected=10 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : seek to 16:5c6b0b28:::7:head 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : cursor()=16:5c6b0b28:::7:head expected=16:5c6b0b28:::7:head 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: > 16:5c6b0b28:::7:head -> 7 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: : entry=7 expected=7 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (242 ms) 2024-02-10T07:42:21.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (101852 ms) 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: split 0/5 -> MIN 16:33333333::::head 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: split 1/5 -> 16:33333333::::head 16:66666666::::head 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: split 2/5 -> 16:66666666::::head 16:99999999::::head 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: split 3/5 -> 16:99999999::::head 16:cccccccc::::head 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: split 4/5 -> 16:cccccccc::::head MAX 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (6009 ms) 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] 7 tests from LibRadosList (108726 ms total) 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-10T07:42:21.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1020 ms) 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset foo1,foo2,foo3 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo1 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo2 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo3 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset foo1,foo4,foo5 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo4 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo5 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo1 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset foo6,foo7 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo7 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: foo6 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns1:foo4 2024-02-10T07:42:21.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns1:foo5 2024-02-10T07:42:21.406 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns2:foo7 2024-02-10T07:42:21.406 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns2:foo6 2024-02-10T07:42:21.406 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: ns1:foo1 2024-02-10T07:42:21.406 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: :foo1 2024-02-10T07:42:21.406 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: :foo2 2024-02-10T07:42:21.406 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: :foo3 2024-02-10T07:42:21.406 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (27 ms) 2024-02-10T07:42:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: pgmap v230: 388 pgs: 388 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:42:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]': finished 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: osdmap e190: 8 total, 8 up, 8 in 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]: dispatch 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]: dispatch 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: pgmap v230: 388 pgs: 388 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]': finished 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: osdmap e190: 8 total, 8 up, 8 in 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]: dispatch 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]: dispatch 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:21.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:21 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: pgmap v230: 388 pgs: 388 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi070-77348-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-19"}]': finished 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi070-77269-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: osdmap e190: 8 total, 8 up, 8 in 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]: dispatch 2024-02-10T07:42:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]: dispatch 2024-02-10T07:42:22.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:22.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:22.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:22.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:21 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:42:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:42:22.763 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]': finished 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: osdmap e191: 8 total, 8 up, 8 in 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/813655773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.36065 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: pgmap v233: 396 pgs: 40 unknown, 356 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.36065 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:22.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: osdmap e192: 8 total, 8 up, 8 in 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]': finished 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: osdmap e191: 8 total, 8 up, 8 in 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/813655773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:22.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.36065 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: pgmap v233: 396 pgs: 40 unknown, 356 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.36065 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: osdmap e192: 8 total, 8 up, 8 in 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:22.766 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:22 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-19", "mode": "writeback"}]': finished 2024-02-10T07:42:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: osdmap e191: 8 total, 8 up, 8 in 2024-02-10T07:42:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/813655773' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.36065 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: pgmap v233: 396 pgs: 40 unknown, 356 active+clean; 583 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi070-77269-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.36065 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi070-77283-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: osdmap e192: 8 total, 8 up, 8 in 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]: dispatch 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:23.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:22 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]': finished 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[29548]: osdmap e193: 8 total, 8 up, 8 in 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]': finished 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[39923]: osdmap e193: 8 total, 8 up, 8 in 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:23 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:23 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:23 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-19"}]': finished 2024-02-10T07:42:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:23 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:23 smithi155 ceph-mon[32688]: osdmap e193: 8 total, 8 up, 8 in 2024-02-10T07:42:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/535780640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:23 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]: dispatch 2024-02-10T07:42:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[29548]: pgmap v236: 364 pgs: 364 active+clean; 595 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 1 op/s 2024-02-10T07:42:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[29548]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[29548]: osdmap e194: 8 total, 8 up, 8 in 2024-02-10T07:42:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3903355449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi070-77283-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[39923]: pgmap v236: 364 pgs: 364 active+clean; 595 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 1 op/s 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[39923]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[39923]: osdmap e194: 8 total, 8 up, 8 in 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3903355449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi070-77283-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:25 smithi155 ceph-mon[32688]: pgmap v236: 364 pgs: 364 active+clean; 595 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 4.5 KiB/s rd, 1 op/s 2024-02-10T07:42:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:25 smithi155 ceph-mon[32688]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi070-77348-2"}]': finished 2024-02-10T07:42:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:25 smithi155 ceph-mon[32688]: osdmap e194: 8 total, 8 up, 8 in 2024-02-10T07:42:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3903355449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi070-77283-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:25 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3903355449' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi070-77283-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: osdmap e195: 8 total, 8 up, 8 in 2024-02-10T07:42:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3855058788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.36077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: from='client.36077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[29548]: osdmap e196: 8 total, 8 up, 8 in 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3903355449' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi070-77283-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: osdmap e195: 8 total, 8 up, 8 in 2024-02-10T07:42:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3855058788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.36077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: from='client.36077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:26 smithi070 ceph-mon[39923]: osdmap e196: 8 total, 8 up, 8 in 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3903355449' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi070-77283-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: osdmap e195: 8 total, 8 up, 8 in 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/161886910' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3855058788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]: dispatch 2024-02-10T07:42:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.36077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:26.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi070-77269-31"}]': finished 2024-02-10T07:42:26.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: from='client.36077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77348-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:26.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:26 smithi155 ceph-mon[32688]: osdmap e196: 8 total, 8 up, 8 in 2024-02-10T07:42:27.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-10T07:42:27.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 1 0 2024-02-10T07:42:27.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 10 0 2024-02-10T07:42:27.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 13 0 2024-02-10T07:42:27.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 7 0 2024-02-10T07:42:27.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 14 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 0 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 15 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 11 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 5 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 8 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 6 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 3 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 4 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 12 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 9 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 2 0 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (42 ms) 2024-02-10T07:42:27.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] 3 tests from LibRadosListEC (1089 ms total) 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3038 ms) 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] 1 test from LibRadosListNP (3038 ms total) 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [----------] Global test environment tear-down 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [==========] 11 tests from 3 test suites ran. (129500 ms total) 2024-02-10T07:42:27.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_list: [ PASSED ] 11 tests. 2024-02-10T07:42:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: pgmap v239: 420 pgs: 96 unknown, 324 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:42:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3855058788' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T07:42:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.36077 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: pgmap v239: 420 pgs: 96 unknown, 324 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3855058788' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.36077 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T07:42:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:27 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: pgmap v239: 420 pgs: 96 unknown, 324 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3855058788' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.36077 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:27 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.36077 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: osdmap e197: 8 total, 8 up, 8 in 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4229988688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[29548]: from='client.36080 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.36077 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: osdmap e197: 8 total, 8 up, 8 in 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4229988688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:28.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:28 smithi070 ceph-mon[39923]: from='client.36080 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.36077 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi070-77348-3","pool2":"test-rados-api-smithi070-77348-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi070-77269-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: osdmap e197: 8 total, 8 up, 8 in 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4229988688' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-mon[32688]: from='client.36080 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:29.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:42:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:42:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[29548]: pgmap v242: 388 pgs: 64 unknown, 324 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]': finished 2024-02-10T07:42:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[29548]: from='client.36080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]: dispatch 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[29548]: osdmap e198: 8 total, 8 up, 8 in 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]: dispatch 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[39923]: pgmap v242: 388 pgs: 64 unknown, 324 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]': finished 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[39923]: from='client.36080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]: dispatch 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[39923]: osdmap e198: 8 total, 8 up, 8 in 2024-02-10T07:42:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]: dispatch 2024-02-10T07:42:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:29 smithi155 ceph-mon[32688]: pgmap v242: 388 pgs: 64 unknown, 324 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-21"}]': finished 2024-02-10T07:42:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:29 smithi155 ceph-mon[32688]: from='client.36080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]: dispatch 2024-02-10T07:42:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:29 smithi155 ceph-mon[32688]: osdmap e198: 8 total, 8 up, 8 in 2024-02-10T07:42:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]: dispatch 2024-02-10T07:42:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]': finished 2024-02-10T07:42:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: osdmap e199: 8 total, 8 up, 8 in 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1650968525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: from='client.36089 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]': finished 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: osdmap e199: 8 total, 8 up, 8 in 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1650968525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: from='client.36089 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi070-77269-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-21", "mode": "writeback"}]': finished 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: osdmap e199: 8 total, 8 up, 8 in 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1650968525' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: from='client.36089 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[39923]: pgmap v245: 428 pgs: 40 unknown, 388 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:42:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[39923]: from='client.36089 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[39923]: osdmap e200: 8 total, 8 up, 8 in 2024-02-10T07:42:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[29548]: pgmap v245: 428 pgs: 40 unknown, 388 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:42:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[29548]: from='client.36089 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[29548]: osdmap e200: 8 total, 8 up, 8 in 2024-02-10T07:42:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:31 smithi155 ceph-mon[32688]: pgmap v245: 428 pgs: 40 unknown, 388 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:42:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:31 smithi155 ceph-mon[32688]: from='client.36089 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:31 smithi155 ceph-mon[32688]: osdmap e200: 8 total, 8 up, 8 in 2024-02-10T07:42:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]: dispatch 2024-02-10T07:42:32.405 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:42:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:42:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]': finished 2024-02-10T07:42:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[29548]: osdmap e201: 8 total, 8 up, 8 in 2024-02-10T07:42:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2429339371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:32.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]': finished 2024-02-10T07:42:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[39923]: osdmap e201: 8 total, 8 up, 8 in 2024-02-10T07:42:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2429339371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:32 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:32 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:32 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-21"}]': finished 2024-02-10T07:42:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:32 smithi155 ceph-mon[32688]: osdmap e201: 8 total, 8 up, 8 in 2024-02-10T07:42:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2429339371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:32 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:33.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[29548]: pgmap v248: 452 pgs: 64 unknown, 388 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:42:33.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:33.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2429339371' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:33.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[29548]: osdmap e202: 8 total, 8 up, 8 in 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[39923]: pgmap v248: 452 pgs: 64 unknown, 388 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2429339371' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[39923]: osdmap e202: 8 total, 8 up, 8 in 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:33.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:33 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:33 smithi155 ceph-mon[32688]: pgmap v248: 452 pgs: 64 unknown, 388 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:42:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:33 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2429339371' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi070-77283-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:33 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:33 smithi155 ceph-mon[32688]: osdmap e202: 8 total, 8 up, 8 in 2024-02-10T07:42:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/524628908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:33 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]: dispatch 2024-02-10T07:42:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: osdmap e203: 8 total, 8 up, 8 in 2024-02-10T07:42:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: osdmap e203: 8 total, 8 up, 8 in 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:34.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:34 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.36083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi070-77269-32"}]': finished 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: osdmap e203: 8 total, 8 up, 8 in 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:34 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[39923]: pgmap v251: 420 pgs: 32 creating+peering, 388 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:42:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[39923]: osdmap e204: 8 total, 8 up, 8 in 2024-02-10T07:42:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[29548]: pgmap v251: 420 pgs: 32 creating+peering, 388 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:42:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[29548]: osdmap e204: 8 total, 8 up, 8 in 2024-02-10T07:42:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:35 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:35 smithi155 ceph-mon[32688]: pgmap v251: 420 pgs: 32 creating+peering, 388 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:42:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:35 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi070-77269-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:35 smithi155 ceph-mon[32688]: osdmap e204: 8 total, 8 up, 8 in 2024-02-10T07:42:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:35 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: osdmap e205: 8 total, 8 up, 8 in 2024-02-10T07:42:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: pgmap v254: 356 pgs: 32 creating+peering, 324 active+clean; 583 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:42:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: osdmap e206: 8 total, 8 up, 8 in 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3660529799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: osdmap e205: 8 total, 8 up, 8 in 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: pgmap v254: 356 pgs: 32 creating+peering, 324 active+clean; 583 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: osdmap e206: 8 total, 8 up, 8 in 2024-02-10T07:42:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:36.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:36.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3660529799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: osdmap e205: 8 total, 8 up, 8 in 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: pgmap v254: 356 pgs: 32 creating+peering, 324 active+clean; 583 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi070-77269-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: osdmap e206: 8 total, 8 up, 8 in 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3660529799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:38.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]': finished 2024-02-10T07:42:38.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3660529799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[29548]: osdmap e207: 8 total, 8 up, 8 in 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]: dispatch 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]: dispatch 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]': finished 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3660529799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[39923]: osdmap e207: 8 total, 8 up, 8 in 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]: dispatch 2024-02-10T07:42:38.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]: dispatch 2024-02-10T07:42:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-23"}]': finished 2024-02-10T07:42:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3660529799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:38 smithi155 ceph-mon[32688]: osdmap e207: 8 total, 8 up, 8 in 2024-02-10T07:42:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]: dispatch 2024-02-10T07:42:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]: dispatch 2024-02-10T07:42:39.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:42:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:42:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: pgmap v257: 396 pgs: 40 unknown, 32 creating+peering, 324 active+clean; 583 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]': finished 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: osdmap e208: 8 total, 8 up, 8 in 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3898587204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[29548]: from='client.36113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: pgmap v257: 396 pgs: 40 unknown, 32 creating+peering, 324 active+clean; 583 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]': finished 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: osdmap e208: 8 total, 8 up, 8 in 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3898587204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:39.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:39 smithi070 ceph-mon[39923]: from='client.36113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: pgmap v257: 396 pgs: 40 unknown, 32 creating+peering, 324 active+clean; 583 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-23", "mode": "writeback"}]': finished 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: osdmap e208: 8 total, 8 up, 8 in 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3898587204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:39 smithi155 ceph-mon[32688]: from='client.36113 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='client.36113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: osdmap e209: 8 total, 8 up, 8 in 2024-02-10T07:42:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='client.36113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: osdmap e209: 8 total, 8 up, 8 in 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:40.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:40.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:40.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='client.36113 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: osdmap e209: 8 total, 8 up, 8 in 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3704578180' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]: dispatch 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: pgmap v260: 420 pgs: 32 creating+peering, 388 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T07:42:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: osdmap e210: 8 total, 8 up, 8 in 2024-02-10T07:42:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/530300612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.36119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: pgmap v260: 420 pgs: 32 creating+peering, 388 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: osdmap e210: 8 total, 8 up, 8 in 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/530300612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:41.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.36119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:41.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:41.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:41 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: pgmap v260: 420 pgs: 32 creating+peering, 388 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.36101 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi070-77269-33"}]': finished 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: osdmap e210: 8 total, 8 up, 8 in 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/530300612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.36119 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:41.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:41 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:42.362 INFO:tasks.workunit.client.0.smithi070.stdout: 0x564b262850d0 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262857d0 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285330 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285440 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285590 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285900 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285d90 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285f10 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262856c0 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286040 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281410 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281520 2024-02-10T07:42:42.363 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26281630 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286610 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286170 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262862a0 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262863f0 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286720 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286920 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286ef0 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286a50 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286b60 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286c90 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26286dc0 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262874e0 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287040 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287170 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262872a0 2024-02-10T07:42:42.364 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262873d0 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287ad0 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287630 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287740 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287850 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287960 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262880c0 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287c20 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287d70 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26287ea0 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262881d0 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288680 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262887d0 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288da0 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288320 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288450 2024-02-10T07:42:42.365 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288900 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288a30 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288b60 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288c90 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289370 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288ed0 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289000 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289130 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289260 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26288560 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262894a0 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262895f0 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289be0 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289740 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289870 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262899a0 2024-02-10T07:42:42.366 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289ad0 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a1b0 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289d10 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289e60 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26289f90 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a300 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a790 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a8e0 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628aed0 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a430 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a560 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628aa30 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ab40 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ac70 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ada0 2024-02-10T07:42:42.367 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b480 2024-02-10T07:42:42.368 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628afe0 2024-02-10T07:42:42.368 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b110 2024-02-10T07:42:42.368 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a670 2024-02-10T07:42:42.368 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b240 2024-02-10T07:42:42.368 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b370 2024-02-10T07:42:42.368 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b5d0 2024-02-10T07:42:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:42:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:42:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]': finished 2024-02-10T07:42:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: from='client.36119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: osdmap e211: 8 total, 8 up, 8 in 2024-02-10T07:42:43.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[29548]: osdmap e212: 8 total, 8 up, 8 in 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]': finished 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: from='client.36119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: osdmap e211: 8 total, 8 up, 8 in 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:42 smithi070 ceph-mon[39923]: osdmap e212: 8 total, 8 up, 8 in 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-23"}]': finished 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: from='client.36119 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi070-77269-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: osdmap e211: 8 total, 8 up, 8 in 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:42 smithi155 ceph-mon[32688]: osdmap e212: 8 total, 8 up, 8 in 2024-02-10T07:42:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: pgmap v263: 452 pgs: 32 unknown, 32 creating+peering, 388 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T07:42:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3752272339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: from='client.36131 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: from='client.36131 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: osdmap e213: 8 total, 8 up, 8 in 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: pgmap v263: 452 pgs: 32 unknown, 32 creating+peering, 388 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3752272339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: from='client.36131 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: from='client.36131 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: osdmap e213: 8 total, 8 up, 8 in 2024-02-10T07:42:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: pgmap v263: 452 pgs: 32 unknown, 32 creating+peering, 388 active+clean; 584 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3752272339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: from='client.36131 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi070-77269-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: from='client.36131 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: osdmap e213: 8 total, 8 up, 8 in 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[29548]: pgmap v266: 492 pgs: 40 creating+peering, 452 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:45.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[29548]: osdmap e214: 8 total, 8 up, 8 in 2024-02-10T07:42:45.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2498407796' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[39923]: pgmap v266: 492 pgs: 40 creating+peering, 452 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[39923]: osdmap e214: 8 total, 8 up, 8 in 2024-02-10T07:42:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2498407796' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:44 smithi155 ceph-mon[32688]: pgmap v266: 492 pgs: 40 creating+peering, 452 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:44 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:44 smithi155 ceph-mon[32688]: osdmap e214: 8 total, 8 up, 8 in 2024-02-10T07:42:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2498407796' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2498407796' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[29548]: osdmap e215: 8 total, 8 up, 8 in 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2498407796' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[39923]: osdmap e215: 8 total, 8 up, 8 in 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2498407796' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi070-77283-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:46 smithi155 ceph-mon[32688]: osdmap e215: 8 total, 8 up, 8 in 2024-02-10T07:42:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:46 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:46.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: pgmap v269: 516 pgs: 32 unknown, 32 creating+peering, 452 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: osdmap e216: 8 total, 8 up, 8 in 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]': finished 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: osdmap e217: 8 total, 8 up, 8 in 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]: dispatch 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]: dispatch 2024-02-10T07:42:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: pgmap v269: 516 pgs: 32 unknown, 32 creating+peering, 452 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: osdmap e216: 8 total, 8 up, 8 in 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]': finished 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: osdmap e217: 8 total, 8 up, 8 in 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:47.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:47 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: pgmap v269: 516 pgs: 32 unknown, 32 creating+peering, 452 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: osdmap e216: 8 total, 8 up, 8 in 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/902838316' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]: dispatch 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-25"}]': finished 2024-02-10T07:42:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.36125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi070-77269-34"}]': finished 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: osdmap e217: 8 total, 8 up, 8 in 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]: dispatch 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]: dispatch 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:47.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:47 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]': finished 2024-02-10T07:42:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: osdmap e218: 8 total, 8 up, 8 in 2024-02-10T07:42:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]': finished 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: osdmap e218: 8 total, 8 up, 8 in 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-25", "mode": "writeback"}]': finished 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi070-77269-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: osdmap e218: 8 total, 8 up, 8 in 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:49.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:42:48 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:42:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:49 smithi070 ceph-mon[29548]: pgmap v272: 452 pgs: 32 creating+peering, 420 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:49 smithi070 ceph-mon[39923]: pgmap v272: 452 pgs: 32 creating+peering, 420 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:49 smithi155 ceph-mon[32688]: pgmap v272: 452 pgs: 32 creating+peering, 420 active+clean; 584 KiB data, 549 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:42:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: osdmap e219: 8 total, 8 up, 8 in 2024-02-10T07:42:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]': finished 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[39923]: osdmap e220: 8 total, 8 up, 8 in 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: osdmap e219: 8 total, 8 up, 8 in 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]': finished 2024-02-10T07:42:50.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:50 smithi070 ceph-mon[29548]: osdmap e220: 8 total, 8 up, 8 in 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: osdmap e219: 8 total, 8 up, 8 in 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]: dispatch 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi070-77269-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-25"}]': finished 2024-02-10T07:42:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:50 smithi155 ceph-mon[32688]: osdmap e220: 8 total, 8 up, 8 in 2024-02-10T07:42:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[29548]: pgmap v275: 388 pgs: 388 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:42:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[29548]: osdmap e221: 8 total, 8 up, 8 in 2024-02-10T07:42:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3326526782' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[29548]: from='client.36149 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[39923]: pgmap v275: 388 pgs: 388 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:42:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[39923]: osdmap e221: 8 total, 8 up, 8 in 2024-02-10T07:42:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3326526782' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:51 smithi070 ceph-mon[39923]: from='client.36149 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:51 smithi155 ceph-mon[32688]: pgmap v275: 388 pgs: 388 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:42:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:51 smithi155 ceph-mon[32688]: osdmap e221: 8 total, 8 up, 8 in 2024-02-10T07:42:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3326526782' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:51 smithi155 ceph-mon[32688]: from='client.36149 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:42:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:42:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: pgmap v278: 364 pgs: 40 unknown, 324 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: from='client.36149 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: osdmap e222: 8 total, 8 up, 8 in 2024-02-10T07:42:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: pgmap v278: 364 pgs: 40 unknown, 324 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: from='client.36149 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: osdmap e222: 8 total, 8 up, 8 in 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:52.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:52 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: pgmap v278: 364 pgs: 40 unknown, 324 active+clean; 583 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: from='client.36149 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi070-77283-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: osdmap e222: 8 total, 8 up, 8 in 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:52 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[39923]: osdmap e223: 8 total, 8 up, 8 in 2024-02-10T07:42:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:54.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[29548]: osdmap e223: 8 total, 8 up, 8 in 2024-02-10T07:42:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:54 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:54 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:54 smithi155 ceph-mon[32688]: osdmap e223: 8 total, 8 up, 8 in 2024-02-10T07:42:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1168581800' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:54 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]: dispatch 2024-02-10T07:42:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: pgmap v281: 356 pgs: 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:42:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: osdmap e224: 8 total, 8 up, 8 in 2024-02-10T07:42:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2481363009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi070-77269-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: pgmap v281: 356 pgs: 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: osdmap e224: 8 total, 8 up, 8 in 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2481363009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi070-77269-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: pgmap v281: 356 pgs: 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.36143 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi070-77269-35"}]': finished 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: osdmap e224: 8 total, 8 up, 8 in 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2481363009' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi070-77269-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:42:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:42:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi070-77269-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: osdmap e225: 8 total, 8 up, 8 in 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi070-77269-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]': finished 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi070-77269-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:56.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: osdmap e225: 8 total, 8 up, 8 in 2024-02-10T07:42:56.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi070-77269-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:56.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:42:56.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:42:56.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]': finished 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi070-77283-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi070-77269-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: osdmap e225: 8 total, 8 up, 8 in 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi070-77269-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:42:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-27"}]': finished 2024-02-10T07:42:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: pgmap v284: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: osdmap e226: 8 total, 8 up, 8 in 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi070-77269-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]': finished 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: osdmap e227: 8 total, 8 up, 8 in 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: pgmap v284: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]: dispatch 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: osdmap e226: 8 total, 8 up, 8 in 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]: dispatch 2024-02-10T07:42:57.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi070-77269-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]': finished 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: osdmap e227: 8 total, 8 up, 8 in 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:57.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:57 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: pgmap v284: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]: dispatch 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: osdmap e226: 8 total, 8 up, 8 in 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]: dispatch 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi070-77269-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-27", "mode": "writeback"}]': finished 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: osdmap e227: 8 total, 8 up, 8 in 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:57 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:42:59.337 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:42:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:42:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[29548]: pgmap v287: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[29548]: osdmap e228: 8 total, 8 up, 8 in 2024-02-10T07:42:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[39923]: pgmap v287: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[39923]: osdmap e228: 8 total, 8 up, 8 in 2024-02-10T07:42:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:42:59 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:59 smithi155 ceph-mon[32688]: pgmap v287: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 554 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:42:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:59 smithi155 ceph-mon[32688]: osdmap e228: 8 total, 8 up, 8 in 2024-02-10T07:42:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:59 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:42:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:42:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:42:59 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[29548]: osdmap e229: 8 total, 8 up, 8 in 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[39923]: osdmap e229: 8 total, 8 up, 8 in 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:43:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:00 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:43:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:00 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:00 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:00 smithi155 ceph-mon[32688]: osdmap e229: 8 total, 8 up, 8 in 2024-02-10T07:43:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:43:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:43:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:00 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]: dispatch 2024-02-10T07:43:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[29548]: pgmap v290: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T07:43:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:43:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]': finished 2024-02-10T07:43:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[29548]: osdmap e230: 8 total, 8 up, 8 in 2024-02-10T07:43:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:43:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[39923]: pgmap v290: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T07:43:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:43:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]': finished 2024-02-10T07:43:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[39923]: osdmap e230: 8 total, 8 up, 8 in 2024-02-10T07:43:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:43:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:01 smithi155 ceph-mon[32688]: pgmap v290: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T07:43:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:01 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:43:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:01 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-27"}]': finished 2024-02-10T07:43:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:01 smithi155 ceph-mon[32688]: osdmap e230: 8 total, 8 up, 8 in 2024-02-10T07:43:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]: dispatch 2024-02-10T07:43:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:43:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:43:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:43:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: osdmap e231: 8 total, 8 up, 8 in 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: pgmap v293: 324 pgs: 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: osdmap e232: 8 total, 8 up, 8 in 2024-02-10T07:43:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi070-77269-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: osdmap e231: 8 total, 8 up, 8 in 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:02.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: pgmap v293: 324 pgs: 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: osdmap e232: 8 total, 8 up, 8 in 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi070-77269-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:02.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:02 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3775010774' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi070-77269-36"}]': finished 2024-02-10T07:43:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: osdmap e231: 8 total, 8 up, 8 in 2024-02-10T07:43:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: pgmap v293: 324 pgs: 324 active+clean; 584 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: osdmap e232: 8 total, 8 up, 8 in 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2329575414' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:03.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi070-77269-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:03.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]: dispatch 2024-02-10T07:43:03.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:03.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:02 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: osdmap e233: 8 total, 8 up, 8 in 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: osdmap e233: 8 total, 8 up, 8 in 2024-02-10T07:43:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:04.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:04 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-46"}]': finished 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: osdmap e233: 8 total, 8 up, 8 in 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:04.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:04 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[29548]: pgmap v296: 356 pgs: 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:43:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi070-77269-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[29548]: osdmap e234: 8 total, 8 up, 8 in 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[39923]: pgmap v296: 356 pgs: 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi070-77269-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[39923]: osdmap e234: 8 total, 8 up, 8 in 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:05 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:05 smithi155 ceph-mon[32688]: pgmap v296: 356 pgs: 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:43:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi070-77269-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:05 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi070-77283-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:05 smithi155 ceph-mon[32688]: osdmap e234: 8 total, 8 up, 8 in 2024-02-10T07:43:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:05 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:06.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: osdmap e235: 8 total, 8 up, 8 in 2024-02-10T07:43:06.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: pgmap v299: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: osdmap e236: 8 total, 8 up, 8 in 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:06.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: osdmap e235: 8 total, 8 up, 8 in 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: pgmap v299: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: osdmap e236: 8 total, 8 up, 8 in 2024-02-10T07:43:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: osdmap e235: 8 total, 8 up, 8 in 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: pgmap v299: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi070-77283-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: osdmap e236: 8 total, 8 up, 8 in 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:06.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]': finished 2024-02-10T07:43:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[29548]: osdmap e237: 8 total, 8 up, 8 in 2024-02-10T07:43:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]: dispatch 2024-02-10T07:43:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]: dispatch 2024-02-10T07:43:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]': finished 2024-02-10T07:43:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[39923]: osdmap e237: 8 total, 8 up, 8 in 2024-02-10T07:43:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]: dispatch 2024-02-10T07:43:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]: dispatch 2024-02-10T07:43:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:07 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-29"}]': finished 2024-02-10T07:43:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:07 smithi155 ceph-mon[32688]: osdmap e237: 8 total, 8 up, 8 in 2024-02-10T07:43:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]: dispatch 2024-02-10T07:43:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]: dispatch 2024-02-10T07:43:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]: dispatch 2024-02-10T07:43:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: pgmap v302: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]': finished 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: osdmap e238: 8 total, 8 up, 8 in 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi070-77269-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: pgmap v302: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]': finished 2024-02-10T07:43:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: osdmap e238: 8 total, 8 up, 8 in 2024-02-10T07:43:08.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:08.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:08.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:08.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:08.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi070-77269-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: pgmap v302: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 576 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2246833798' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi070-77269-37"}]': finished 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-29", "mode": "writeback"}]': finished 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: osdmap e238: 8 total, 8 up, 8 in 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi070-77269-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:09.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:43:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:43:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi070-77269-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: osdmap e239: 8 total, 8 up, 8 in 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi070-77269-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi070-77269-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: osdmap e239: 8 total, 8 up, 8 in 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi070-77269-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi070-77269-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: osdmap e239: 8 total, 8 up, 8 in 2024-02-10T07:43:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi070-77269-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1664984035' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]: dispatch 2024-02-10T07:43:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:11.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: pgmap v305: 356 pgs: 356 active+clean; 13 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-10T07:43:11.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:11.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: osdmap e240: 8 total, 8 up, 8 in 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: pgmap v305: 356 pgs: 356 active+clean; 13 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: osdmap e240: 8 total, 8 up, 8 in 2024-02-10T07:43:11.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:11.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:11 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: pgmap v305: 356 pgs: 356 active+clean; 13 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi070-77283-47"}]': finished 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: osdmap e240: 8 total, 8 up, 8 in 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:11 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:43:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:43:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi070-77269-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: osdmap e241: 8 total, 8 up, 8 in 2024-02-10T07:43:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi070-77269-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: osdmap e241: 8 total, 8 up, 8 in 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi070-77269-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi070-77283-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: osdmap e241: 8 total, 8 up, 8 in 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: pgmap v308: 364 pgs: 8 unknown, 356 active+clean; 13 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-10T07:43:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: osdmap e242: 8 total, 8 up, 8 in 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]': finished 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: osdmap e243: 8 total, 8 up, 8 in 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: pgmap v308: 364 pgs: 8 unknown, 356 active+clean; 13 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: osdmap e242: 8 total, 8 up, 8 in 2024-02-10T07:43:13.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:13.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:13.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:13.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]': finished 2024-02-10T07:43:13.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: osdmap e243: 8 total, 8 up, 8 in 2024-02-10T07:43:13.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: pgmap v308: 364 pgs: 8 unknown, 356 active+clean; 13 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: osdmap e242: 8 total, 8 up, 8 in 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]: dispatch 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi070-77283-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-29"}]': finished 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: osdmap e243: 8 total, 8 up, 8 in 2024-02-10T07:43:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[29548]: pgmap v311: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 8.5 KiB/s rd, 1024 KiB/s wr, 16 op/s 2024-02-10T07:43:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[29548]: osdmap e244: 8 total, 8 up, 8 in 2024-02-10T07:43:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[39923]: pgmap v311: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 8.5 KiB/s rd, 1024 KiB/s wr, 16 op/s 2024-02-10T07:43:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[39923]: osdmap e244: 8 total, 8 up, 8 in 2024-02-10T07:43:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:15 smithi155 ceph-mon[32688]: pgmap v311: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 8.5 KiB/s rd, 1024 KiB/s wr, 16 op/s 2024-02-10T07:43:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:15 smithi155 ceph-mon[32688]: osdmap e244: 8 total, 8 up, 8 in 2024-02-10T07:43:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]: dispatch 2024-02-10T07:43:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: osdmap e245: 8 total, 8 up, 8 in 2024-02-10T07:43:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:16.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[29548]: osdmap e246: 8 total, 8 up, 8 in 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: osdmap e245: 8 total, 8 up, 8 in 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:16.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:16.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:16.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:16.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:16 smithi070 ceph-mon[39923]: osdmap e246: 8 total, 8 up, 8 in 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/75035336' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi070-77269-38"}]': finished 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: osdmap e245: 8 total, 8 up, 8 in 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi070-77269-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:16.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:16 smithi155 ceph-mon[32688]: osdmap e246: 8 total, 8 up, 8 in 2024-02-10T07:43:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: pgmap v314: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[29548]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: pgmap v314: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:17 smithi070 ceph-mon[39923]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: pgmap v314: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2161313918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]: dispatch 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:17 smithi155 ceph-mon[32688]: from='client.36188 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi070-77283-48"}]': finished 2024-02-10T07:43:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: osdmap e247: 8 total, 8 up, 8 in 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: pgmap v317: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]': finished 2024-02-10T07:43:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: osdmap e248: 8 total, 8 up, 8 in 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: osdmap e247: 8 total, 8 up, 8 in 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: pgmap v317: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]': finished 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: osdmap e248: 8 total, 8 up, 8 in 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]: dispatch 2024-02-10T07:43:18.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]: dispatch 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: osdmap e247: 8 total, 8 up, 8 in 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: pgmap v317: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi070-77269-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-31"}]': finished 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi070-77283-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: osdmap e248: 8 total, 8 up, 8 in 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]: dispatch 2024-02-10T07:43:19.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]: dispatch 2024-02-10T07:43:19.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:43:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:43:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:19 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]': finished 2024-02-10T07:43:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:19 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]': finished 2024-02-10T07:43:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:19 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-31", "mode": "writeback"}]': finished 2024-02-10T07:43:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: osdmap e249: 8 total, 8 up, 8 in 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: pgmap v320: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: osdmap e250: 8 total, 8 up, 8 in 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: osdmap e249: 8 total, 8 up, 8 in 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: pgmap v320: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: osdmap e250: 8 total, 8 up, 8 in 2024-02-10T07:43:20.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:20.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:20.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:20.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:20 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: osdmap e249: 8 total, 8 up, 8 in 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: pgmap v320: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi070-77283-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: osdmap e250: 8 total, 8 up, 8 in 2024-02-10T07:43:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]: dispatch 2024-02-10T07:43:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:20 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]': finished 2024-02-10T07:43:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[29548]: osdmap e251: 8 total, 8 up, 8 in 2024-02-10T07:43:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]': finished 2024-02-10T07:43:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[39923]: osdmap e251: 8 total, 8 up, 8 in 2024-02-10T07:43:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:21 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:21 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:21 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-31"}]': finished 2024-02-10T07:43:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:21 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:21 smithi155 ceph-mon[32688]: osdmap e251: 8 total, 8 up, 8 in 2024-02-10T07:43:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2266821212' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:21 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]: dispatch 2024-02-10T07:43:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:43:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:43:23.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: pgmap v323: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: osdmap e252: 8 total, 8 up, 8 in 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: pgmap v323: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: osdmap e252: 8 total, 8 up, 8 in 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:23.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:23.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:23.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:22 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: pgmap v323: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.26158 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi070-77269-39"}]': finished 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: osdmap e252: 8 total, 8 up, 8 in 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:23.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:22 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: osdmap e253: 8 total, 8 up, 8 in 2024-02-10T07:43:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: osdmap e253: 8 total, 8 up, 8 in 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:24.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:24 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi070-77269-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: osdmap e253: 8 total, 8 up, 8 in 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3436981428' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:24 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]: dispatch 2024-02-10T07:43:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: pgmap v326: 356 pgs: 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: osdmap e254: 8 total, 8 up, 8 in 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: pgmap v326: 356 pgs: 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: osdmap e254: 8 total, 8 up, 8 in 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:25.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: pgmap v326: 356 pgs: 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi070-77283-49"}]': finished 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: osdmap e254: 8 total, 8 up, 8 in 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:25.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:25.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: osdmap e255: 8 total, 8 up, 8 in 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: osdmap e255: 8 total, 8 up, 8 in 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:26.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:26 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi070-77269-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi070-77283-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: osdmap e255: 8 total, 8 up, 8 in 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:26 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:27.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[39923]: pgmap v329: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:27.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]': finished 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]: dispatch 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[39923]: osdmap e256: 8 total, 8 up, 8 in 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]: dispatch 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[29548]: pgmap v329: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]': finished 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]: dispatch 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[29548]: osdmap e256: 8 total, 8 up, 8 in 2024-02-10T07:43:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:27 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]: dispatch 2024-02-10T07:43:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:27 smithi155 ceph-mon[32688]: pgmap v329: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:27 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-33"}]': finished 2024-02-10T07:43:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]: dispatch 2024-02-10T07:43:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:27 smithi155 ceph-mon[32688]: osdmap e256: 8 total, 8 up, 8 in 2024-02-10T07:43:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:27 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]: dispatch 2024-02-10T07:43:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]': finished 2024-02-10T07:43:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: osdmap e257: 8 total, 8 up, 8 in 2024-02-10T07:43:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]': finished 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: osdmap e257: 8 total, 8 up, 8 in 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:28.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:28 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi070-77283-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-33", "mode": "writeback"}]': finished 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: osdmap e257: 8 total, 8 up, 8 in 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:28 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:29.367 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:43:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: pgmap v332: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: osdmap e258: 8 total, 8 up, 8 in 2024-02-10T07:43:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: pgmap v332: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: osdmap e258: 8 total, 8 up, 8 in 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:29 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: pgmap v332: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: osdmap e258: 8 total, 8 up, 8 in 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1550965398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]: dispatch 2024-02-10T07:43:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:29 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]: dispatch 2024-02-10T07:43:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]': finished 2024-02-10T07:43:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: osdmap e259: 8 total, 8 up, 8 in 2024-02-10T07:43:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi070-77269-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]': finished 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: osdmap e259: 8 total, 8 up, 8 in 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:30.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:30.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi070-77269-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-33"}]': finished 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi070-77269-40"}]': finished 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: osdmap e259: 8 total, 8 up, 8 in 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi070-77269-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: pgmap v335: 356 pgs: 356 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-10T07:43:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi070-77269-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: osdmap e260: 8 total, 8 up, 8 in 2024-02-10T07:43:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi070-77269-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: pgmap v335: 356 pgs: 356 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi070-77269-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: osdmap e260: 8 total, 8 up, 8 in 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi070-77269-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:31 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: pgmap v335: 356 pgs: 356 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi070-77269-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: osdmap e260: 8 total, 8 up, 8 in 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2998512235' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi070-77269-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:31 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]: dispatch 2024-02-10T07:43:32.385 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:43:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:43:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[29548]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[29548]: osdmap e261: 8 total, 8 up, 8 in 2024-02-10T07:43:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:32.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[39923]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:32.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[39923]: osdmap e261: 8 total, 8 up, 8 in 2024-02-10T07:43:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:32.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:32 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:32 smithi155 ceph-mon[32688]: from='client.36212 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi070-77283-50"}]': finished 2024-02-10T07:43:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:32 smithi155 ceph-mon[32688]: osdmap e261: 8 total, 8 up, 8 in 2024-02-10T07:43:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:32 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: pgmap v338: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi070-77269-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: osdmap e262: 8 total, 8 up, 8 in 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/914312700' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: pgmap v338: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi070-77269-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: osdmap e262: 8 total, 8 up, 8 in 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/914312700' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:33.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: pgmap v338: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi070-77269-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: osdmap e262: 8 total, 8 up, 8 in 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/914312700' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[29548]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[29548]: osdmap e263: 8 total, 8 up, 8 in 2024-02-10T07:43:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[39923]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[39923]: osdmap e263: 8 total, 8 up, 8 in 2024-02-10T07:43:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:34 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:34 smithi155 ceph-mon[32688]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi070-77283-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:34 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:34 smithi155 ceph-mon[32688]: osdmap e263: 8 total, 8 up, 8 in 2024-02-10T07:43:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:34 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: pgmap v341: 396 pgs: 396 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: osdmap e264: 8 total, 8 up, 8 in 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]: dispatch 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]: dispatch 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]': finished 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: osdmap e265: 8 total, 8 up, 8 in 2024-02-10T07:43:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi070-77283-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: pgmap v341: 396 pgs: 396 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: osdmap e264: 8 total, 8 up, 8 in 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:35.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]': finished 2024-02-10T07:43:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: osdmap e265: 8 total, 8 up, 8 in 2024-02-10T07:43:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:35.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi070-77283-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: pgmap v341: 396 pgs: 396 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:43:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: osdmap e264: 8 total, 8 up, 8 in 2024-02-10T07:43:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]: dispatch 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]: dispatch 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-35", "mode": "writeback"}]': finished 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: osdmap e265: 8 total, 8 up, 8 in 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]: dispatch 2024-02-10T07:43:35.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi070-77283-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: pgmap v344: 356 pgs: 356 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: osdmap e266: 8 total, 8 up, 8 in 2024-02-10T07:43:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: pgmap v344: 356 pgs: 356 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: osdmap e266: 8 total, 8 up, 8 in 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:37.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:37 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: pgmap v344: 356 pgs: 356 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3062194438' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi070-77269-41"}]': finished 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: osdmap e266: 8 total, 8 up, 8 in 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:37 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi070-77283-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[39923]: osdmap e267: 8 total, 8 up, 8 in 2024-02-10T07:43:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi070-77283-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[29548]: osdmap e267: 8 total, 8 up, 8 in 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi070-77283-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:38 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi070-77269-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:38 smithi155 ceph-mon[32688]: osdmap e267: 8 total, 8 up, 8 in 2024-02-10T07:43:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:38 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:39.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:43:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: pgmap v347: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: osdmap e268: 8 total, 8 up, 8 in 2024-02-10T07:43:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: osdmap e269: 8 total, 8 up, 8 in 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: pgmap v347: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: osdmap e268: 8 total, 8 up, 8 in 2024-02-10T07:43:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: osdmap e269: 8 total, 8 up, 8 in 2024-02-10T07:43:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: pgmap v347: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: osdmap e268: 8 total, 8 up, 8 in 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi070-77269-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: osdmap e269: 8 total, 8 up, 8 in 2024-02-10T07:43:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:41.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: pgmap v350: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T07:43:41.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: osdmap e270: 8 total, 8 up, 8 in 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: pgmap v350: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:41.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:41.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: osdmap e270: 8 total, 8 up, 8 in 2024-02-10T07:43:41.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:41.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:41.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:41 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: pgmap v350: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: osdmap e270: 8 total, 8 up, 8 in 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]: dispatch 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:41 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:42.417 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:43:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:43:42.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:42.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:42.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: osdmap e271: 8 total, 8 up, 8 in 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:42.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: osdmap e271: 8 total, 8 up, 8 in 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:42.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:42 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3547315528' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi070-77283-52"}]': finished 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: osdmap e271: 8 total, 8 up, 8 in 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]: dispatch 2024-02-10T07:43:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:42.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:42.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:42.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:42 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:43.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: sta api_aio: [ OK ] LibRadosAioEC.SimpleWrite (6744 ms) 2024-02-10T07:43:43.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-02-10T07:43:43.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7011 ms) 2024-02-10T07:43:43.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-10T07:43:43.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6995 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7037 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7019 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7060 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7039 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6856 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7018 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7260 ms) 2024-02-10T07:43:43.432 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6741 ms) 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7022 ms) 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6975 ms) 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7035 ms) 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7274 ms) 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6786 ms) 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111872 ms total) 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [----------] Global test environment tear-down 2024-02-10T07:43:43.433 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (205572 ms total) 2024-02-10T07:43:43.434 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio: [ PASSED ] 42 tests. 2024-02-10T07:43:43.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: pgmap v353: 356 pgs: 356 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T07:43:43.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:43.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:43.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: osdmap e272: 8 total, 8 up, 8 in 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: pgmap v353: 356 pgs: 356 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:43.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:43.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: osdmap e272: 8 total, 8 up, 8 in 2024-02-10T07:43:43.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:43.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:43.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:43 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: pgmap v353: 356 pgs: 356 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-35"}]': finished 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: osdmap e272: 8 total, 8 up, 8 in 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3537006851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:43 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]: dispatch 2024-02-10T07:43:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:44 smithi070 ceph-mon[29548]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:44 smithi070 ceph-mon[29548]: osdmap e273: 8 total, 8 up, 8 in 2024-02-10T07:43:44.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:44 smithi070 ceph-mon[39923]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:44.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:44 smithi070 ceph-mon[39923]: osdmap e273: 8 total, 8 up, 8 in 2024-02-10T07:43:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:44 smithi155 ceph-mon[32688]: from='client.36236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi070-77269-42"}]': finished 2024-02-10T07:43:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:44 smithi155 ceph-mon[32688]: osdmap e273: 8 total, 8 up, 8 in 2024-02-10T07:43:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[29548]: pgmap v356: 324 pgs: 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:43:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[29548]: osdmap e274: 8 total, 8 up, 8 in 2024-02-10T07:43:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[39923]: pgmap v356: 324 pgs: 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:43:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[39923]: osdmap e274: 8 total, 8 up, 8 in 2024-02-10T07:43:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:45 smithi155 ceph-mon[32688]: pgmap v356: 324 pgs: 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:43:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:45 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi070-77283-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:45 smithi155 ceph-mon[32688]: osdmap e274: 8 total, 8 up, 8 in 2024-02-10T07:43:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:46.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:46.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: osdmap e275: 8 total, 8 up, 8 in 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: osdmap e276: 8 total, 8 up, 8 in 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: osdmap e275: 8 total, 8 up, 8 in 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:46.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:46.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: osdmap e276: 8 total, 8 up, 8 in 2024-02-10T07:43:46.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:46.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:46.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:46.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:46 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: osdmap e275: 8 total, 8 up, 8 in 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: osdmap e276: 8 total, 8 up, 8 in 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:46 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: pgmap v359: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]': finished 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: osdmap e277: 8 total, 8 up, 8 in 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]: dispatch 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]: dispatch 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: pgmap v359: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]': finished 2024-02-10T07:43:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: osdmap e277: 8 total, 8 up, 8 in 2024-02-10T07:43:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]: dispatch 2024-02-10T07:43:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:47.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]: dispatch 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: pgmap v359: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-37"}]': finished 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: osdmap e277: 8 total, 8 up, 8 in 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]: dispatch 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2953847439' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]: dispatch 2024-02-10T07:43:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]: dispatch 2024-02-10T07:43:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]': finished 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: osdmap e278: 8 total, 8 up, 8 in 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:48.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]': finished 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: osdmap e278: 8 total, 8 up, 8 in 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:48.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.36242 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi070-77283-53"}]': finished 2024-02-10T07:43:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-37", "mode": "writeback"}]': finished 2024-02-10T07:43:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: osdmap e278: 8 total, 8 up, 8 in 2024-02-10T07:43:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:48.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:48.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:48.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:48.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:49.451 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:43:49.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: pgmap v362: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:49.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:49.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:49.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: osdmap e279: 8 total, 8 up, 8 in 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: pgmap v362: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: osdmap e279: 8 total, 8 up, 8 in 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:49.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:49.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: pgmap v362: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi070-77283-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: osdmap e279: 8 total, 8 up, 8 in 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]: dispatch 2024-02-10T07:43:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:50 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]': finished 2024-02-10T07:43:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:50 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]': finished 2024-02-10T07:43:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:43:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:50 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-37"}]': finished 2024-02-10T07:43:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[29548]: pgmap v365: 356 pgs: 356 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T07:43:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[29548]: osdmap e280: 8 total, 8 up, 8 in 2024-02-10T07:43:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[29548]: osdmap e281: 8 total, 8 up, 8 in 2024-02-10T07:43:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[39923]: pgmap v365: 356 pgs: 356 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T07:43:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[39923]: osdmap e280: 8 total, 8 up, 8 in 2024-02-10T07:43:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:51 smithi070 ceph-mon[39923]: osdmap e281: 8 total, 8 up, 8 in 2024-02-10T07:43:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:51 smithi155 ceph-mon[32688]: pgmap v365: 356 pgs: 356 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T07:43:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:51 smithi155 ceph-mon[32688]: osdmap e280: 8 total, 8 up, 8 in 2024-02-10T07:43:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:51 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi070-77283-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:51 smithi155 ceph-mon[32688]: osdmap e281: 8 total, 8 up, 8 in 2024-02-10T07:43:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:43:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:43:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:43:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[29548]: pgmap v368: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:43:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[29548]: osdmap e282: 8 total, 8 up, 8 in 2024-02-10T07:43:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[39923]: pgmap v368: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:43:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[39923]: osdmap e282: 8 total, 8 up, 8 in 2024-02-10T07:43:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:53 smithi155 ceph-mon[32688]: pgmap v368: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:43:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:53 smithi155 ceph-mon[32688]: osdmap e282: 8 total, 8 up, 8 in 2024-02-10T07:43:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[29548]: osdmap e283: 8 total, 8 up, 8 in 2024-02-10T07:43:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[39923]: osdmap e283: 8 total, 8 up, 8 in 2024-02-10T07:43:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:54 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:54 smithi155 ceph-mon[32688]: osdmap e283: 8 total, 8 up, 8 in 2024-02-10T07:43:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:43:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:54 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: pgmap v371: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: osdmap e284: 8 total, 8 up, 8 in 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: pgmap v371: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: osdmap e284: 8 total, 8 up, 8 in 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:55.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: pgmap v371: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: osdmap e284: 8 total, 8 up, 8 in 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2883716461' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]: dispatch 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:43:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:43:56.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:56.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]': finished 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: osdmap e285: 8 total, 8 up, 8 in 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]: dispatch 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]: dispatch 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi070-77283-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]': finished 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: osdmap e285: 8 total, 8 up, 8 in 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]: dispatch 2024-02-10T07:43:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]: dispatch 2024-02-10T07:43:56.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:56.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:56.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi070-77283-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi070-77283-54"}]': finished 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-39"}]': finished 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: osdmap e285: 8 total, 8 up, 8 in 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]: dispatch 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]: dispatch 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi070-77283-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:43:57.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: pgmap v374: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:57.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:57.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]': finished 2024-02-10T07:43:57.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi070-77283-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: osdmap e286: 8 total, 8 up, 8 in 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi070-77283-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: pgmap v374: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]': finished 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi070-77283-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: osdmap e286: 8 total, 8 up, 8 in 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi070-77283-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:57 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: pgmap v374: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-39", "mode": "writeback"}]': finished 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi070-77283-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: osdmap e286: 8 total, 8 up, 8 in 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi070-77283-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:57 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:43:58.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:58.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:58.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[29548]: osdmap e287: 8 total, 8 up, 8 in 2024-02-10T07:43:58.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:58.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:58.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:58.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[39923]: osdmap e287: 8 total, 8 up, 8 in 2024-02-10T07:43:58.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:58 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:58.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:58 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:43:58.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:58.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:58 smithi155 ceph-mon[32688]: osdmap e287: 8 total, 8 up, 8 in 2024-02-10T07:43:58.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:58 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]: dispatch 2024-02-10T07:43:59.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-02-10T07:43:59.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-10T07:43:59.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-10T07:43:59.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: seed 77569 2024-02-10T07:43:59.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (400 ms) 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (15111 ms) 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (65 ms) 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (15576 ms total) 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (8152 ms) 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8119 ms) 2024-02-10T07:43:59.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9746 ms) 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: my_snaps [3] 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: my_snaps [4,3] 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: my_snaps [5,4,3] 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: promoting some heads 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: promoting from clones for snap 3 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: waiting for scrubs... 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: done waiting 2024-02-10T07:43:59.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46019 ms) 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10013 ms) 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8007 ms) 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8033 ms) 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8065 ms) 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10283 ms) 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8627 ms) 2024-02-10T07:43:59.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10024 ms) 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12999 ms) 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7980 ms) 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8053 ms) 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13282 ms) 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7671 ms) 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-10T07:43:59.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8039 ms) 2024-02-10T07:43:59.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:43:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:43:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[29548]: pgmap v377: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi070-77283-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]': finished 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[29548]: osdmap e288: 8 total, 8 up, 8 in 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[39923]: pgmap v377: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi070-77283-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]': finished 2024-02-10T07:43:59.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:43:59 smithi070 ceph-mon[39923]: osdmap e288: 8 total, 8 up, 8 in 2024-02-10T07:43:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:59 smithi155 ceph-mon[32688]: pgmap v377: 356 pgs: 356 active+clean; 4.6 MiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:43:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:59 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:43:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi070-77283-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:43:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:59 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-39"}]': finished 2024-02-10T07:43:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:43:59 smithi155 ceph-mon[32688]: osdmap e288: 8 total, 8 up, 8 in 2024-02-10T07:44:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:00 smithi070 ceph-mon[29548]: osdmap e289: 8 total, 8 up, 8 in 2024-02-10T07:44:00.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:00.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:00 smithi070 ceph-mon[39923]: osdmap e289: 8 total, 8 up, 8 in 2024-02-10T07:44:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:00 smithi155 ceph-mon[32688]: osdmap e289: 8 total, 8 up, 8 in 2024-02-10T07:44:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[29548]: pgmap v380: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T07:44:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[29548]: osdmap e290: 8 total, 8 up, 8 in 2024-02-10T07:44:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[39923]: pgmap v380: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[39923]: osdmap e290: 8 total, 8 up, 8 in 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:01.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:01 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:01 smithi155 ceph-mon[32688]: pgmap v380: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T07:44:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:01 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:01 smithi155 ceph-mon[32688]: osdmap e290: 8 total, 8 up, 8 in 2024-02-10T07:44:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:44:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:01 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:02.395 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:44:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:44:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:02.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[29548]: osdmap e291: 8 total, 8 up, 8 in 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[39923]: osdmap e291: 8 total, 8 up, 8 in 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:02.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:02 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:44:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:02 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:02 smithi155 ceph-mon[32688]: osdmap e291: 8 total, 8 up, 8 in 2024-02-10T07:44:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]: dispatch 2024-02-10T07:44:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:02.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:02 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:03.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: pgmap v383: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: osdmap e292: 8 total, 8 up, 8 in 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:03.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: pgmap v383: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: osdmap e292: 8 total, 8 up, 8 in 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:03.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:03 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: pgmap v383: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2939402570' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi070-77283-55"}]': finished 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: osdmap e292: 8 total, 8 up, 8 in 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:03.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:03 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:04.376 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSerted 0x564b2628baa0 2024-02-10T07:44:04.376 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628bbf0 2024-02-10T07:44:04.376 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628c1c0 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b720 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b830 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628b960 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628c2f0 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628bd20 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628be70 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628bf80 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628c0b0 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628a0a0 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628c440 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628c8d0 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ca20 2024-02-10T07:44:04.377 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285a30 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26285b80 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d010 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628cb70 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628cc80 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628cdb0 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628cec0 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d600 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d160 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d2b0 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d3e0 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d750 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628dbe0 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628dd10 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628c550 2024-02-10T07:44:04.378 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628c6a0 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e2c0 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628de20 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628df50 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e080 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e1b0 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e8b0 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e410 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e540 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e670 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e7a0 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ee60 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628e9c0 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628eb10 2024-02-10T07:44:04.379 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ec20 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ed50 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f430 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628ef90 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f0a0 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f1b0 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f2c0 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628fa20 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f580 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f690 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f7e0 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628fb50 2024-02-10T07:44:04.380 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290000 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290130 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290700 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628fca0 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628fdb0 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628fee0 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628f910 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290260 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290390 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262904c0 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262905d0 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290cf0 2024-02-10T07:44:04.381 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290850 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290960 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290ab0 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290e20 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262912b0 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262913e0 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d880 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628d9b0 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262919b0 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26291510 2024-02-10T07:44:04.382 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26291660 2024-02-10T07:44:04.383 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26291770 2024-02-10T07:44:04.383 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262918a0 2024-02-10T07:44:04.383 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26291f80 2024-02-10T07:44:04.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:44:04.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:04.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: osdmap e293: 8 total, 8 up, 8 in 2024-02-10T07:44:04.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: osdmap e293: 8 total, 8 up, 8 in 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:04.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:04.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:04.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:04 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:44:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi070-77283-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: osdmap e293: 8 total, 8 up, 8 in 2024-02-10T07:44:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:05.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:05.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:04 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[29548]: pgmap v386: 356 pgs: 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:44:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[29548]: osdmap e294: 8 total, 8 up, 8 in 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[29548]: osdmap e295: 8 total, 8 up, 8 in 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[39923]: pgmap v386: 356 pgs: 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[39923]: osdmap e294: 8 total, 8 up, 8 in 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:44:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:05 smithi070 ceph-mon[39923]: osdmap e295: 8 total, 8 up, 8 in 2024-02-10T07:44:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:05 smithi155 ceph-mon[32688]: pgmap v386: 356 pgs: 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:44:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:05 smithi155 ceph-mon[32688]: osdmap e294: 8 total, 8 up, 8 in 2024-02-10T07:44:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:05 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:44:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:05 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi070-77283-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:05 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:44:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:05 smithi155 ceph-mon[32688]: osdmap e295: 8 total, 8 up, 8 in 2024-02-10T07:44:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[29548]: pgmap v389: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[29548]: osdmap e296: 8 total, 8 up, 8 in 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[39923]: pgmap v389: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[39923]: osdmap e296: 8 total, 8 up, 8 in 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]: dispatch 2024-02-10T07:44:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:07 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:07 smithi155 ceph-mon[32688]: pgmap v389: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:07 smithi155 ceph-mon[32688]: osdmap e296: 8 total, 8 up, 8 in 2024-02-10T07:44:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]: dispatch 2024-02-10T07:44:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]: dispatch 2024-02-10T07:44:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:07 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]': finished 2024-02-10T07:44:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[29548]: osdmap e297: 8 total, 8 up, 8 in 2024-02-10T07:44:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:08.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]': finished 2024-02-10T07:44:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[39923]: osdmap e297: 8 total, 8 up, 8 in 2024-02-10T07:44:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:08.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:08 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-41"}]': finished 2024-02-10T07:44:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:08 smithi155 ceph-mon[32688]: osdmap e297: 8 total, 8 up, 8 in 2024-02-10T07:44:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:08 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:09.429 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:44:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:44:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[39923]: pgmap v392: 356 pgs: 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[39923]: osdmap e298: 8 total, 8 up, 8 in 2024-02-10T07:44:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:09.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[29548]: pgmap v392: 356 pgs: 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:09.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:09.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[29548]: osdmap e298: 8 total, 8 up, 8 in 2024-02-10T07:44:09.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:09.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:09 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:09 smithi155 ceph-mon[32688]: pgmap v392: 356 pgs: 356 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:09 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:09 smithi155 ceph-mon[32688]: osdmap e298: 8 total, 8 up, 8 in 2024-02-10T07:44:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3456045478' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:09 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]: dispatch 2024-02-10T07:44:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: osdmap e299: 8 total, 8 up, 8 in 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: osdmap e300: 8 total, 8 up, 8 in 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: osdmap e299: 8 total, 8 up, 8 in 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: osdmap e300: 8 total, 8 up, 8 in 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:10 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.36260 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi070-77283-56"}]': finished 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: osdmap e299: 8 total, 8 up, 8 in 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi070-77283-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: osdmap e300: 8 total, 8 up, 8 in 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:10.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:10 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[39923]: pgmap v395: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:12.441 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi070-77569-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T07:44:12.441 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:12.441 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:12.441 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[29548]: pgmap v395: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi070-77569-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:11 smithi155 ceph-mon[32688]: pgmap v395: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi070-77569-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T07:44:12.442 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:12.443 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:12.443 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:44:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:12.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T07:44:12.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: osdmap e301: 8 total, 8 up, 8 in 2024-02-10T07:44:12.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: pgmap v398: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]': finished 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: osdmap e302: 8 total, 8 up, 8 in 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: osdmap e301: 8 total, 8 up, 8 in 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: pgmap v398: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:12.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]': finished 2024-02-10T07:44:12.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: osdmap e302: 8 total, 8 up, 8 in 2024-02-10T07:44:12.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:12.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: osdmap e301: 8 total, 8 up, 8 in 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: pgmap v398: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi070-77283-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_count","val": "8"}]': finished 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: osdmap e302: 8 total, 8 up, 8 in 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[29548]: osdmap e303: 8 total, 8 up, 8 in 2024-02-10T07:44:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T07:44:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T07:44:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[39923]: osdmap e303: 8 total, 8 up, 8 in 2024-02-10T07:44:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T07:44:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T07:44:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:13 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:13 smithi155 ceph-mon[32688]: osdmap e303: 8 total, 8 up, 8 in 2024-02-10T07:44:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T07:44:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc:tRead 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: ok, hit_set contains 263:602f83fe:::foo:head 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9048 ms) 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg_num = 32 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 0 ls 1707551055,0 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 1 ls 1707551055,0 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 2 ls 1707551055,0 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 3 ls 1707551055,0 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 4 ls 1707551055,0 2024-02-10T07:44:14.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 5 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 6 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 7 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 8 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 9 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 10 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 11 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 12 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 13 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 14 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 15 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 16 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 17 ls 1707551055,0 2024-02-10T07:44:14.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 18 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 19 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 20 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 21 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 22 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 23 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 24 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 25 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 26 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 27 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 28 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 29 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 30 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg 31 ls 1707551055,0 2024-02-10T07:44:14.661 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: pg_num = 32 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6cac518f:::0:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:02547ec2:::1:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f905c69b:::2:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:cfc208b3:::3:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d83876eb:::4:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b29083e3:::5:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c4fdafeb:::6:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:5c6b0b28:::7:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:bd63b0f1:::8:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:e960b815:::9:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:52ea6a34:::10:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:89d3ae78:::11:head 2024-02-10T07:44:14.662 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:de5d7c5f:::12:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:566253c9:::13:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:62a1935d:::14:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:863748b0:::15:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:3958e169:::16:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:4d4dabf9:::17:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:8391935d:::18:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:28883081:::19:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:69259c59:::20:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:4bdb80b7:::21:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a11c5d71:::22:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:271af37b:::23:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:95b121be:::24:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:58d1031b:::25:head 2024-02-10T07:44:14.663 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:0a050783:::26:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c709704c:::27:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:cbe56eaf:::28:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:86b4b162:::29:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:70d89383:::30:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:dd450c7c:::31:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6d5729b1:::32:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c388f3fb:::33:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:56cfea31:::34:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:9dbc1bf7:::35:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:40b74ccd:::36:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:4d5aaf42:::37:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:920f362c:::38:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6cc53222:::39:head 2024-02-10T07:44:14.664 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:9cad833f:::40:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:1ea84d41:::41:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c4480ef6:::42:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a694361e:::43:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d1bd33e9:::44:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:ddc2cd5d:::45:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:2b782207:::46:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:7b187fca:::47:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:90ecdf6f:::48:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a5ed95fe:::49:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:ea0eaa55:::50:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f33ef17b:::51:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a0d1b2f6:::52:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:60c5229e:::53:head 2024-02-10T07:44:14.665 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:edcbc575:::54:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:102cf253:::55:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:efb7fb0b:::56:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:50d0a326:::57:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d4dc5daf:::58:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:3a130462:::59:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:ec87ed71:::60:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d5bc9454:::61:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:3ddfe313:::62:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:7c2816b9:::63:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:47e00e4d:::64:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c6410c18:::65:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b48ed237:::66:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:cd63ad31:::67:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b179e92b:::68:head 2024-02-10T07:44:14.666 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:0d9f741a:::69:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6d3352ae:::70:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c6d5c19e:::71:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:bc4729c3:::72:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:77e930b9:::73:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:0abeecfd:::74:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b7c37e15:::75:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b6378398:::76:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:02bd68de:::77:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:cc795d2d:::78:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:630d4fea:::79:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:e0d29ef5:::80:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:fd6f13d2:::81:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:606461d5:::82:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:eadbdc43:::83:head 2024-02-10T07:44:14.667 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:8761d0bb:::84:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:9ef0186f:::85:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:e0d41294:::86:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:961de695:::87:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:1423148f:::88:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:633a8fa2:::89:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a8653809:::90:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:3dac8b33:::91:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:35aad435:::92:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f6dcc343:::93:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:dbbdad87:::94:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:1cb48ce0:::95:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:03cd461c:::96:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:17a4ea99:::97:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:9993c9a7:::98:head 2024-02-10T07:44:14.668 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6394211c:::99:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:94c7ae57:::100:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6fdee5bb:::101:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:9a477fd1:::102:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:eb850916:::103:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:affc56b9:::104:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b42dc814:::105:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f319f8f0:::106:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:9a40b9de:::107:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:8b524f28:::108:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:e3de589f:::109:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:90f90a5b:::110:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a7b4f1d7:::111:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:af51766e:::112:head 2024-02-10T07:44:14.669 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b6f90bd1:::113:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:e0261208:::114:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c9569ef7:::115:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:61bebe50:::116:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:fe93412b:::117:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d3d38bee:::118:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:3100ba0c:::119:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d0560ada:::120:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f0ea8b35:::121:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:766f231a:::122:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a07a2582:::123:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:bd7c6b3a:::124:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:fb2ddaff:::125:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:4408e1fe:::126:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:ee1df7a7:::127:head 2024-02-10T07:44:14.670 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c3002909:::128:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:4f48ffa9:::129:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:edf38733:::130:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c08425c0:::131:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:5f902d98:::132:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:41ea2c93:::133:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:813cee13:::134:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:0131818d:::135:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:26ba5a85:::136:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:381b8a5a:::137:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:28797e47:::138:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:bfca7f22:::139:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:36807075:::140:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:80b03975:::141:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:5c15709b:::142:head 2024-02-10T07:44:14.671 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f39ea15e:::143:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:ea992956:::144:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:48887b1c:::145:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:9f24a9dd:::146:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:987f100b:::147:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d2dd3581:::148:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:7fed1808:::149:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c80b70e9:::150:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:85ed90f9:::151:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:36428b24:::152:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d044c34a:::153:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:7c18bf58:::154:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d1c21232:::155:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a7a3c575:::156:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:87da0633:::157:head 2024-02-10T07:44:14.672 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d5ac3822:::158:head 2024-02-10T07:44:14.673 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:3f20522d:::159:head 2024-02-10T07:44:14.673 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6ca26563:::160:head 2024-02-10T07:44:14.673 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:532ce135:::161:head 2024-02-10T07:44:14.673 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:c78863e6:::162:head 2024-02-10T07:44:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: pgmap v401: 364 pgs: 3 creating+activating, 361 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T07:44:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: osdmap e304: 8 total, 8 up, 8 in 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi070-77569-43","var": "pg_num","format": "json"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: pgmap v401: 364 pgs: 3 creating+activating, 361 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: osdmap e304: 8 total, 8 up, 8 in 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi070-77569-43","var": "pg_num","format": "json"}]: dispatch 2024-02-10T07:44:15.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:15.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:15.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]: dispatch 2024-02-10T07:44:15.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]: dispatch 2024-02-10T07:44:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: pgmap v401: 364 pgs: 3 creating+activating, 361 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T07:44:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: osdmap e304: 8 total, 8 up, 8 in 2024-02-10T07:44:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi070-77569-43","var": "pg_num","format": "json"}]: dispatch 2024-02-10T07:44:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:15.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:15.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]: dispatch 2024-02-10T07:44:15.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]: dispatch 2024-02-10T07:44:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]': finished 2024-02-10T07:44:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[39923]: osdmap e305: 8 total, 8 up, 8 in 2024-02-10T07:44:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[39923]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[39923]: osdmap e306: 8 total, 8 up, 8 in 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]': finished 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[29548]: osdmap e305: 8 total, 8 up, 8 in 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[29548]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:16 smithi070 ceph-mon[29548]: osdmap e306: 8 total, 8 up, 8 in 2024-02-10T07:44:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:16 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:16 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-43"}]': finished 2024-02-10T07:44:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:16 smithi155 ceph-mon[32688]: osdmap e305: 8 total, 8 up, 8 in 2024-02-10T07:44:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2479164686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:16 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]: dispatch 2024-02-10T07:44:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:16 smithi155 ceph-mon[32688]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi070-77283-57"}]': finished 2024-02-10T07:44:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:16 smithi155 ceph-mon[32688]: osdmap e306: 8 total, 8 up, 8 in 2024-02-10T07:44:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: pgmap v404: 356 pgs: 356 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: osdmap e307: 8 total, 8 up, 8 in 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: pgmap v404: 356 pgs: 356 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: osdmap e307: 8 total, 8 up, 8 in 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:17.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:17 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: pgmap v404: 356 pgs: 356 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi070-77283-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: osdmap e307: 8 total, 8 up, 8 in 2024-02-10T07:44:18.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:18.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:18.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:18.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:17 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:19.441 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:44:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:44:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[29548]: pgmap v407: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[29548]: osdmap e308: 8 total, 8 up, 8 in 2024-02-10T07:44:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[39923]: pgmap v407: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[39923]: osdmap e308: 8 total, 8 up, 8 in 2024-02-10T07:44:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:19 smithi155 ceph-mon[32688]: pgmap v407: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:19 smithi155 ceph-mon[32688]: osdmap e308: 8 total, 8 up, 8 in 2024-02-10T07:44:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:19 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[29548]: osdmap e309: 8 total, 8 up, 8 in 2024-02-10T07:44:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:44:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:44:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[39923]: osdmap e309: 8 total, 8 up, 8 in 2024-02-10T07:44:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:44:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:44:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:20 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi070-77283-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:20 smithi155 ceph-mon[32688]: osdmap e309: 8 total, 8 up, 8 in 2024-02-10T07:44:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:44:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:44:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[29548]: pgmap v410: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]': finished 2024-02-10T07:44:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[29548]: osdmap e310: 8 total, 8 up, 8 in 2024-02-10T07:44:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]': finished 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[29548]: osdmap e311: 8 total, 8 up, 8 in 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[39923]: pgmap v410: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]': finished 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[39923]: osdmap e310: 8 total, 8 up, 8 in 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]': finished 2024-02-10T07:44:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:21 smithi070 ceph-mon[39923]: osdmap e311: 8 total, 8 up, 8 in 2024-02-10T07:44:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:21 smithi155 ceph-mon[32688]: pgmap v410: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_count","val": "3"}]': finished 2024-02-10T07:44:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:21 smithi155 ceph-mon[32688]: osdmap e310: 8 total, 8 up, 8 in 2024-02-10T07:44:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:44:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:44:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_period","val": "3"}]': finished 2024-02-10T07:44:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:21 smithi155 ceph-mon[32688]: osdmap e311: 8 total, 8 up, 8 in 2024-02-10T07:44:22.461 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:44:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: osdmap e312: 8 total, 8 up, 8 in 2024-02-10T07:44:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: osdmap e312: 8 total, 8 up, 8 in 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: osdmap e312: 8 total, 8 up, 8 in 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2171845894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:44:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: pgmap v413: 356 pgs: 356 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:44:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[29548]: osdmap e313: 8 total, 8 up, 8 in 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: pgmap v413: 356 pgs: 356 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:23.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T07:44:23.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:23.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:23 smithi070 ceph-mon[39923]: osdmap e313: 8 total, 8 up, 8 in 2024-02-10T07:44:23.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: pgmap v413: 356 pgs: 356 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]: dispatch 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi070-77283-58"}]': finished 2024-02-10T07:44:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:23 smithi155 ceph-mon[32688]: osdmap e313: 8 total, 8 up, 8 in 2024-02-10T07:44:24.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:24.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:24.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:24.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:24.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:24 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:24.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:24.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:24 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:44:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[29548]: pgmap v416: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[29548]: osdmap e314: 8 total, 8 up, 8 in 2024-02-10T07:44:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[39923]: pgmap v416: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[39923]: osdmap e314: 8 total, 8 up, 8 in 2024-02-10T07:44:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:25 smithi155 ceph-mon[32688]: pgmap v416: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:25 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:25 smithi155 ceph-mon[32688]: osdmap e314: 8 total, 8 up, 8 in 2024-02-10T07:44:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:25 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[29548]: osdmap e315: 8 total, 8 up, 8 in 2024-02-10T07:44:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[29548]: pgmap v419: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[29548]: osdmap e316: 8 total, 8 up, 8 in 2024-02-10T07:44:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[39923]: osdmap e315: 8 total, 8 up, 8 in 2024-02-10T07:44:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[39923]: pgmap v419: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:26 smithi070 ceph-mon[39923]: osdmap e316: 8 total, 8 up, 8 in 2024-02-10T07:44:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:26 smithi155 ceph-mon[32688]: osdmap e315: 8 total, 8 up, 8 in 2024-02-10T07:44:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:26 smithi155 ceph-mon[32688]: pgmap v419: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:26 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi070-77283-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:26 smithi155 ceph-mon[32688]: osdmap e316: 8 total, 8 up, 8 in 2024-02-10T07:44:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:27 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:27 smithi070 ceph-mon[29548]: osdmap e317: 8 total, 8 up, 8 in 2024-02-10T07:44:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:27 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:27 smithi070 ceph-mon[39923]: osdmap e317: 8 total, 8 up, 8 in 2024-02-10T07:44:28.004 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:27 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:27 smithi155 ceph-mon[32688]: osdmap e317: 8 total, 8 up, 8 in 2024-02-10T07:44:29.472 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:44:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:44:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[29548]: pgmap v422: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[29548]: osdmap e318: 8 total, 8 up, 8 in 2024-02-10T07:44:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[39923]: pgmap v422: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[39923]: osdmap e318: 8 total, 8 up, 8 in 2024-02-10T07:44:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:29 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:29 smithi155 ceph-mon[32688]: pgmap v422: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:29 smithi155 ceph-mon[32688]: osdmap e318: 8 total, 8 up, 8 in 2024-02-10T07:44:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:29.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:29 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:30.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:7462ddf6:::o.RoundTripAppendPP (3250 ms) 2024-02-10T07:44:30.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-10T07:44:30.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2765 ms) 2024-02-10T07:44:30.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-10T07:44:30.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2996 ms) 2024-02-10T07:44:30.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-10T07:44:30.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3007 ms) 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: setting pool EIO 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: max_success 100, min_failed 101 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4026 ms) 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3003 ms) 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (125496 ms total) 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3010 ms) 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-10T07:44:30.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9304 ms) 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14655 ms) 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3007 ms) 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29976 ms total) 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3245 ms) 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3245 ms total) 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-10T07:44:30.494 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13773 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6977 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7018 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7048 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3011 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7018 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6915 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7029 ms) 2024-02-10T07:44:30.495 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7040 ms) 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7021 ms) 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7012 ms) 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7022 ms) 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-10T07:44:30.496 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7034 ms) 2024-02-10T07:44:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[29548]: osdmap e319: 8 total, 8 up, 8 in 2024-02-10T07:44:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[39923]: osdmap e319: 8 total, 8 up, 8 in 2024-02-10T07:44:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:30 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:30 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:30 smithi155 ceph-mon[32688]: osdmap e319: 8 total, 8 up, 8 in 2024-02-10T07:44:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/343335474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:30 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]: dispatch 2024-02-10T07:44:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:31 smithi070 ceph-mon[29548]: pgmap v425: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:31 smithi070 ceph-mon[29548]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:31 smithi070 ceph-mon[29548]: osdmap e320: 8 total, 8 up, 8 in 2024-02-10T07:44:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:31 smithi070 ceph-mon[39923]: pgmap v425: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:31 smithi070 ceph-mon[39923]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:31 smithi070 ceph-mon[39923]: osdmap e320: 8 total, 8 up, 8 in 2024-02-10T07:44:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:31 smithi155 ceph-mon[32688]: pgmap v425: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:31 smithi155 ceph-mon[32688]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi070-77283-59"}]': finished 2024-02-10T07:44:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:31 smithi155 ceph-mon[32688]: osdmap e320: 8 total, 8 up, 8 in 2024-02-10T07:44:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:44:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:44:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:32 smithi155 ceph-mon[32688]: osdmap e321: 8 total, 8 up, 8 in 2024-02-10T07:44:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2658751476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:32.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:32 smithi070 ceph-mon[29548]: osdmap e321: 8 total, 8 up, 8 in 2024-02-10T07:44:32.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2658751476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:32 smithi070 ceph-mon[39923]: osdmap e321: 8 total, 8 up, 8 in 2024-02-10T07:44:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2658751476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:33 smithi155 ceph-mon[32688]: pgmap v428: 388 pgs: 32 unknown, 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:33 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2658751476' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:33 smithi155 ceph-mon[32688]: osdmap e322: 8 total, 8 up, 8 in 2024-02-10T07:44:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[29548]: pgmap v428: 388 pgs: 32 unknown, 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2658751476' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[29548]: osdmap e322: 8 total, 8 up, 8 in 2024-02-10T07:44:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[39923]: pgmap v428: 388 pgs: 32 unknown, 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2658751476' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi070-77283-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:33 smithi070 ceph-mon[39923]: osdmap e322: 8 total, 8 up, 8 in 2024-02-10T07:44:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[29548]: osdmap e323: 8 total, 8 up, 8 in 2024-02-10T07:44:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[39923]: osdmap e323: 8 total, 8 up, 8 in 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:34 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:34 smithi155 ceph-mon[32688]: osdmap e323: 8 total, 8 up, 8 in 2024-02-10T07:44:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:34 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:34 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:34 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: pgmap v431: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T07:44:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: osdmap e324: 8 total, 8 up, 8 in 2024-02-10T07:44:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]: dispatch 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]: dispatch 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: pgmap v431: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: osdmap e324: 8 total, 8 up, 8 in 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:35.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:35.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]: dispatch 2024-02-10T07:44:35.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]: dispatch 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: pgmap v431: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi070-77283-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: osdmap e324: 8 total, 8 up, 8 in 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]: dispatch 2024-02-10T07:44:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]: dispatch 2024-02-10T07:44:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]': finished 2024-02-10T07:44:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:36 smithi070 ceph-mon[29548]: osdmap e325: 8 total, 8 up, 8 in 2024-02-10T07:44:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]': finished 2024-02-10T07:44:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:36 smithi070 ceph-mon[39923]: osdmap e325: 8 total, 8 up, 8 in 2024-02-10T07:44:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-45"}]': finished 2024-02-10T07:44:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:36 smithi155 ceph-mon[32688]: osdmap e325: 8 total, 8 up, 8 in 2024-02-10T07:44:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:37 smithi155 ceph-mon[32688]: pgmap v434: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T07:44:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:37 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:37 smithi155 ceph-mon[32688]: osdmap e326: 8 total, 8 up, 8 in 2024-02-10T07:44:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:37 smithi070 ceph-mon[29548]: pgmap v434: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T07:44:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:37 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:37 smithi070 ceph-mon[29548]: osdmap e326: 8 total, 8 up, 8 in 2024-02-10T07:44:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:37 smithi070 ceph-mon[39923]: pgmap v434: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T07:44:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:37 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi070-77283-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:37 smithi070 ceph-mon[39923]: osdmap e326: 8 total, 8 up, 8 in 2024-02-10T07:44:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[29548]: osdmap e327: 8 total, 8 up, 8 in 2024-02-10T07:44:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[39923]: osdmap e327: 8 total, 8 up, 8 in 2024-02-10T07:44:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:38 smithi155 ceph-mon[32688]: osdmap e327: 8 total, 8 up, 8 in 2024-02-10T07:44:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:39.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:44:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: pgmap v437: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: osdmap e328: 8 total, 8 up, 8 in 2024-02-10T07:44:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: osdmap e329: 8 total, 8 up, 8 in 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: pgmap v437: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: osdmap e328: 8 total, 8 up, 8 in 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: osdmap e329: 8 total, 8 up, 8 in 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:39.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:39.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: pgmap v437: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: osdmap e328: 8 total, 8 up, 8 in 2024-02-10T07:44:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: osdmap e329: 8 total, 8 up, 8 in 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1382930883' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]: dispatch 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[29548]: pgmap v440: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[29548]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]': finished 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[29548]: osdmap e330: 8 total, 8 up, 8 in 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]: dispatch 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]: dispatch 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[39923]: pgmap v440: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[39923]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]': finished 2024-02-10T07:44:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[39923]: osdmap e330: 8 total, 8 up, 8 in 2024-02-10T07:44:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]: dispatch 2024-02-10T07:44:40.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]: dispatch 2024-02-10T07:44:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:40 smithi155 ceph-mon[32688]: pgmap v440: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:44:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:40 smithi155 ceph-mon[32688]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi070-77283-61"}]': finished 2024-02-10T07:44:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-47"}]': finished 2024-02-10T07:44:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:40 smithi155 ceph-mon[32688]: osdmap e330: 8 total, 8 up, 8 in 2024-02-10T07:44:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]: dispatch 2024-02-10T07:44:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]: dispatch 2024-02-10T07:44:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]': finished 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: osdmap e331: 8 total, 8 up, 8 in 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]': finished 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: osdmap e331: 8 total, 8 up, 8 in 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:41 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-47", "mode": "writeback"}]': finished 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: osdmap e331: 8 total, 8 up, 8 in 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:44:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:41 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:44:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:44:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[29548]: pgmap v443: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:44:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:44:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[29548]: osdmap e332: 8 total, 8 up, 8 in 2024-02-10T07:44:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[39923]: pgmap v443: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:44:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:44:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[39923]: osdmap e332: 8 total, 8 up, 8 in 2024-02-10T07:44:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:44.004 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:43 smithi155 ceph-mon[32688]: pgmap v443: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:44:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:44:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:43 smithi155 ceph-mon[32688]: osdmap e332: 8 total, 8 up, 8 in 2024-02-10T07:44:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:44:44.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:44.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[29548]: osdmap e333: 8 total, 8 up, 8 in 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[39923]: osdmap e333: 8 total, 8 up, 8 in 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:44:44.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:44:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:44 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi070-77283-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:44:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:44 smithi155 ceph-mon[32688]: osdmap e333: 8 total, 8 up, 8 in 2024-02-10T07:44:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:44:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:44 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:44:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:44:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: pgmap v446: 364 pgs: 8 creating+peering, 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: osdmap e334: 8 total, 8 up, 8 in 2024-02-10T07:44:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:44:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: osdmap e335: 8 total, 8 up, 8 in 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: pgmap v446: 364 pgs: 8 creating+peering, 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: osdmap e334: 8 total, 8 up, 8 in 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:44:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:44:45.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: osdmap e335: 8 total, 8 up, 8 in 2024-02-10T07:44:45.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:44:45.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:44:45.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:45.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:45 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: pgmap v446: 364 pgs: 8 creating+peering, 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: osdmap e334: 8 total, 8 up, 8 in 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: osdmap e335: 8 total, 8 up, 8 in 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:44:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:46.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:45 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: pgmap v449: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: osdmap e336: 8 total, 8 up, 8 in 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: pgmap v449: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: osdmap e336: 8 total, 8 up, 8 in 2024-02-10T07:44:47.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:47.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:47.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:44:47.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: pgmap v449: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: osdmap e336: 8 total, 8 up, 8 in 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1175966678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]: dispatch 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:44:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:44:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: osdmap e337: 8 total, 8 up, 8 in 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi070-77283-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[29548]: pgmap v452: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: osdmap e337: 8 total, 8 up, 8 in 2024-02-10T07:44:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi070-77283-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:48.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:48 smithi070 ceph-mon[39923]: pgmap v452: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: from='client.36293 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi070-77283-62"}]': finished 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: osdmap e337: 8 total, 8 up, 8 in 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi070-77283-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:48 smithi155 ceph-mon[32688]: pgmap v452: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:49.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:44:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi070-77283-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:44:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[39923]: osdmap e338: 8 total, 8 up, 8 in 2024-02-10T07:44:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi070-77283-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi070-77283-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[29548]: osdmap e338: 8 total, 8 up, 8 in 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi070-77283-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:49 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi070-77283-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:44:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-mon[32688]: osdmap e338: 8 total, 8 up, 8 in 2024-02-10T07:44:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi070-77283-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:49 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:44:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]': finished 2024-02-10T07:44:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: osdmap e339: 8 total, 8 up, 8 in 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: pgmap v455: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi070-77283-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[29548]: osdmap e340: 8 total, 8 up, 8 in 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]': finished 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: osdmap e339: 8 total, 8 up, 8 in 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: pgmap v455: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:44:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi070-77283-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:51.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:50 smithi070 ceph-mon[39923]: osdmap e340: 8 total, 8 up, 8 in 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]': finished 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: osdmap e339: 8 total, 8 up, 8 in 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-47"}]: dispatch 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: pgmap v455: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi070-77283-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:50 smithi155 ceph-mon[32688]: osdmap e340: 8 total, 8 up, 8 in 2024-02-10T07:44:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:44:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[29548]: osdmap e341: 8 total, 8 up, 8 in 2024-02-10T07:44:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[29548]: pgmap v458: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:44:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[39923]: osdmap e341: 8 total, 8 up, 8 in 2024-02-10T07:44:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[39923]: pgmap v458: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:44:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:52 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:52 smithi155 ceph-mon[32688]: osdmap e341: 8 total, 8 up, 8 in 2024-02-10T07:44:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:52 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:44:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:52 smithi155 ceph-mon[32688]: pgmap v458: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T07:44:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:52 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:44:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[29548]: osdmap e342: 8 total, 8 up, 8 in 2024-02-10T07:44:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[39923]: osdmap e342: 8 total, 8 up, 8 in 2024-02-10T07:44:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:44:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:53 smithi155 ceph-mon[32688]: osdmap e342: 8 total, 8 up, 8 in 2024-02-10T07:44:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:44:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: osdmap e343: 8 total, 8 up, 8 in 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: pgmap v461: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: osdmap e343: 8 total, 8 up, 8 in 2024-02-10T07:44:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:54.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:44:54.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:44:54.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: pgmap v461: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:54.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: osdmap e343: 8 total, 8 up, 8 in 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]: dispatch 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: pgmap v461: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2132869490' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi070-77283-63"}]': finished 2024-02-10T07:44:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]': finished 2024-02-10T07:44:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: osdmap e344: 8 total, 8 up, 8 in 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]: dispatch 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]: dispatch 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi070-77283-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]': finished 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi070-77283-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: osdmap e345: 8 total, 8 up, 8 in 2024-02-10T07:44:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi070-77283-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]': finished 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: osdmap e344: 8 total, 8 up, 8 in 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]: dispatch 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]: dispatch 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi070-77283-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]': finished 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi070-77283-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: osdmap e345: 8 total, 8 up, 8 in 2024-02-10T07:44:55.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi070-77283-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-49"}]': finished 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: osdmap e344: 8 total, 8 up, 8 in 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]: dispatch 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]: dispatch 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi070-77283-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-49", "mode": "readproxy"}]': finished 2024-02-10T07:44:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi070-77283-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:44:56.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: osdmap e345: 8 total, 8 up, 8 in 2024-02-10T07:44:56.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi070-77283-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:44:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:56 smithi070 ceph-mon[29548]: pgmap v464: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:57.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:56 smithi070 ceph-mon[39923]: pgmap v464: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:56 smithi155 ceph-mon[32688]: pgmap v464: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:44:58.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:57 smithi070 ceph-mon[39923]: osdmap e346: 8 total, 8 up, 8 in 2024-02-10T07:44:58.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi070-77283-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:44:58.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:57 smithi070 ceph-mon[39923]: osdmap e347: 8 total, 8 up, 8 in 2024-02-10T07:44:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:57 smithi070 ceph-mon[29548]: osdmap e346: 8 total, 8 up, 8 in 2024-02-10T07:44:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi070-77283-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:44:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:57 smithi070 ceph-mon[29548]: osdmap e347: 8 total, 8 up, 8 in 2024-02-10T07:44:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:57 smithi155 ceph-mon[32688]: osdmap e346: 8 total, 8 up, 8 in 2024-02-10T07:44:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi070-77283-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:44:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:57 smithi155 ceph-mon[32688]: osdmap e347: 8 total, 8 up, 8 in 2024-02-10T07:44:59.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:58 smithi155 ceph-mon[32688]: pgmap v467: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:59.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:58 smithi155 ceph-mon[32688]: osdmap e348: 8 total, 8 up, 8 in 2024-02-10T07:44:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:58 smithi070 ceph-mon[29548]: pgmap v467: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:58 smithi070 ceph-mon[29548]: osdmap e348: 8 total, 8 up, 8 in 2024-02-10T07:44:59.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:58 smithi070 ceph-mon[39923]: pgmap v467: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:44:59.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:58 smithi070 ceph-mon[39923]: osdmap e348: 8 total, 8 up, 8 in 2024-02-10T07:44:59.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:44:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:45:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[29548]: osdmap e349: 8 total, 8 up, 8 in 2024-02-10T07:45:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:45:00.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:00.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[39923]: osdmap e349: 8 total, 8 up, 8 in 2024-02-10T07:45:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:44:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:45:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:59 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:59 smithi155 ceph-mon[32688]: osdmap e349: 8 total, 8 up, 8 in 2024-02-10T07:45:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:44:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:45:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[29548]: pgmap v470: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:45:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:45:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[29548]: osdmap e350: 8 total, 8 up, 8 in 2024-02-10T07:45:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:45:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[39923]: pgmap v470: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:45:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:45:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[39923]: osdmap e350: 8 total, 8 up, 8 in 2024-02-10T07:45:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:45:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:00 smithi155 ceph-mon[32688]: pgmap v470: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:45:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:45:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:00 smithi155 ceph-mon[32688]: osdmap e350: 8 total, 8 up, 8 in 2024-02-10T07:45:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]: dispatch 2024-02-10T07:45:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:45:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:45:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[29548]: osdmap e351: 8 total, 8 up, 8 in 2024-02-10T07:45:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi070-77283-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[29548]: pgmap v473: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[39923]: osdmap e351: 8 total, 8 up, 8 in 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi070-77283-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:02 smithi070 ceph-mon[39923]: pgmap v473: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:45:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3558978850' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi070-77283-64"}]': finished 2024-02-10T07:45:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:02 smithi155 ceph-mon[32688]: osdmap e351: 8 total, 8 up, 8 in 2024-02-10T07:45:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi070-77283-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:02 smithi155 ceph-mon[32688]: pgmap v473: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:45:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi070-77283-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:45:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:03 smithi070 ceph-mon[29548]: osdmap e352: 8 total, 8 up, 8 in 2024-02-10T07:45:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi070-77283-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi070-77283-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:45:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:03 smithi070 ceph-mon[39923]: osdmap e352: 8 total, 8 up, 8 in 2024-02-10T07:45:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi070-77283-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi070-77283-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:45:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:03 smithi155 ceph-mon[32688]: osdmap e352: 8 total, 8 up, 8 in 2024-02-10T07:45:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi070-77283-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[29548]: osdmap e353: 8 total, 8 up, 8 in 2024-02-10T07:45:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[29548]: pgmap v476: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi070-77283-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[29548]: osdmap e354: 8 total, 8 up, 8 in 2024-02-10T07:45:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[39923]: osdmap e353: 8 total, 8 up, 8 in 2024-02-10T07:45:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[39923]: pgmap v476: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi070-77283-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:04 smithi070 ceph-mon[39923]: osdmap e354: 8 total, 8 up, 8 in 2024-02-10T07:45:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:04 smithi155 ceph-mon[32688]: osdmap e353: 8 total, 8 up, 8 in 2024-02-10T07:45:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:04 smithi155 ceph-mon[32688]: pgmap v476: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi070-77283-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:04 smithi155 ceph-mon[32688]: osdmap e354: 8 total, 8 up, 8 in 2024-02-10T07:45:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[29548]: osdmap e355: 8 total, 8 up, 8 in 2024-02-10T07:45:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[29548]: pgmap v479: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[39923]: osdmap e355: 8 total, 8 up, 8 in 2024-02-10T07:45:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:06 smithi070 ceph-mon[39923]: pgmap v479: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:06 smithi155 ceph-mon[32688]: osdmap e355: 8 total, 8 up, 8 in 2024-02-10T07:45:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:06 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:06 smithi155 ceph-mon[32688]: pgmap v479: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:45:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[29548]: osdmap e356: 8 total, 8 up, 8 in 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[39923]: osdmap e356: 8 total, 8 up, 8 in 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:07 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:45:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:07 smithi155 ceph-mon[32688]: osdmap e356: 8 total, 8 up, 8 in 2024-02-10T07:45:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:08.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP163:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:5d165639:::164:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f43765fc:::165:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b4c720e9:::166:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:e694b040:::167:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:afa38db2:::168:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:77ba9f53:::169:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:87495034:::170:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:7c96bf0e:::171:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:dbe346cc:::172:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:e943ec24:::173:head 2024-02-10T07:45:08.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f97a9c0c:::174:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:6f26e74d:::175:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:4f95e106:::176:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:0e6f2f8f:::177:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:05db05f1:::178:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:38a78d66:::179:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:d095610b:::180:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:a1a9d709:::181:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:1e5d39db:::182:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:f7df4fb9:::183:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:03a7f161:::184:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:ba70721e:::185:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:28e5662d:::186:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:973d52de:::187:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:4303eb1c:::188:head 2024-02-10T07:45:08.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:b990b48e:::189:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:29b8165b:::190:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:3547f197:::191:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:7e260936:::192:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:1abec7b1:::193:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:10fdda93:::194:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:15817eea:::195:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:770bab57:::196:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:ed9e13e7:::197:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:71471a8f:::198:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: checking for 265:10fb1d02:::199:head 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8009 ms) 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,0 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: first is 1707551064 2024-02-10T07:45:08.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,1707551069,1707551070,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,1707551069,1707551070,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,1707551069,1707551070,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,1707551069,1707551070,1707551072,1707551073,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,1707551069,1707551070,1707551072,1707551073,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551064,1707551066,1707551067,1707551069,1707551070,1707551072,1707551073,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551067,1707551069,1707551070,1707551072,1707551073,1707551075,1707551076,0 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: first now 1707551067, trimmed 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20092 ms) 2024-02-10T07:45:08.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-10T07:45:08.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: foo0 2024-02-10T07:45:08.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-10T07:45:08.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14038 ms) 2024-02-10T07:45:08.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]': finished 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: osdmap e357: 8 total, 8 up, 8 in 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:09.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: pgmap v482: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:09.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:08 smithi155 ceph-mon[32688]: osdmap e358: 8 total, 8 up, 8 in 2024-02-10T07:45:09.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:45:09.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]': finished 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: osdmap e357: 8 total, 8 up, 8 in 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: pgmap v482: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:09.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[29548]: osdmap e358: 8 total, 8 up, 8 in 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]': finished 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: osdmap e357: 8 total, 8 up, 8 in 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]: dispatch 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-49"}]: dispatch 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: pgmap v482: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3013368860' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi070-77283-65"}]': finished 2024-02-10T07:45:09.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:08 smithi070 ceph-mon[39923]: osdmap e358: 8 total, 8 up, 8 in 2024-02-10T07:45:09.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:45:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: osdmap e359: 8 total, 8 up, 8 in 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: osdmap e359: 8 total, 8 up, 8 in 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:10.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:09 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi070-77283-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: osdmap e359: 8 total, 8 up, 8 in 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:10.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:10.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:09 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:10 smithi070 ceph-mon[29548]: pgmap v485: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:11.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:10 smithi070 ceph-mon[39923]: pgmap v485: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:10 smithi155 ceph-mon[32688]: pgmap v485: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:12.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:45:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:45:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: osdmap e360: 8 total, 8 up, 8 in 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: osdmap e361: 8 total, 8 up, 8 in 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: osdmap e360: 8 total, 8 up, 8 in 2024-02-10T07:45:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:45:12.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:12.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:45:12.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: osdmap e361: 8 total, 8 up, 8 in 2024-02-10T07:45:12.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:12.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: osdmap e360: 8 total, 8 up, 8 in 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi070-77283-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: osdmap e361: 8 total, 8 up, 8 in 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[39923]: pgmap v488: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]': finished 2024-02-10T07:45:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[39923]: osdmap e362: 8 total, 8 up, 8 in 2024-02-10T07:45:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]: dispatch 2024-02-10T07:45:13.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]: dispatch 2024-02-10T07:45:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[29548]: pgmap v488: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]': finished 2024-02-10T07:45:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[29548]: osdmap e362: 8 total, 8 up, 8 in 2024-02-10T07:45:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]: dispatch 2024-02-10T07:45:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]: dispatch 2024-02-10T07:45:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:13 smithi155 ceph-mon[32688]: pgmap v488: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:45:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-51"}]': finished 2024-02-10T07:45:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:13 smithi155 ceph-mon[32688]: osdmap e362: 8 total, 8 up, 8 in 2024-02-10T07:45:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]: dispatch 2024-02-10T07:45:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]: dispatch 2024-02-10T07:45:14.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:45:14.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]': finished 2024-02-10T07:45:14.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[39923]: osdmap e363: 8 total, 8 up, 8 in 2024-02-10T07:45:14.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]': finished 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[29548]: osdmap e363: 8 total, 8 up, 8 in 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:45:14.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:14 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:45:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:14 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:45:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:14 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-51", "mode": "writeback"}]': finished 2024-02-10T07:45:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:14 smithi155 ceph-mon[32688]: osdmap e363: 8 total, 8 up, 8 in 2024-02-10T07:45:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:14 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:45:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:14 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ 2 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3018 ms) 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7026 ms) 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7019 ms) 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7025 ms) 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7012 ms) 2024-02-10T07:45:15.646 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7022 ms) 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7029 ms) 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139069 ms total) 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (297786 ms total) 2024-02-10T07:45:15.647 INFO:tasks.workunit.client.0.smithi070.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-02-10T07:45:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: pgmap v491: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:45:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:45:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: osdmap e364: 8 total, 8 up, 8 in 2024-02-10T07:45:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: pgmap v491: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: osdmap e364: 8 total, 8 up, 8 in 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:45:15.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:15 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: pgmap v491: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: osdmap e364: 8 total, 8 up, 8 in 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/812285870' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:45:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:15 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]: dispatch 2024-02-10T07:45:16.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:45:16.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[29548]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:16.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[29548]: osdmap e365: 8 total, 8 up, 8 in 2024-02-10T07:45:16.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[29548]: pgmap v494: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[39923]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[39923]: osdmap e365: 8 total, 8 up, 8 in 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:45:16.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:16 smithi070 ceph-mon[39923]: pgmap v494: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:45:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:16 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:45:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:16 smithi155 ceph-mon[32688]: from='client.36317 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi070-77283-66"}]': finished 2024-02-10T07:45:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:16 smithi155 ceph-mon[32688]: osdmap e365: 8 total, 8 up, 8 in 2024-02-10T07:45:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:45:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:16 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:45:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:16 smithi155 ceph-mon[32688]: pgmap v494: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:45:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:45:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:45:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[29548]: osdmap e366: 8 total, 8 up, 8 in 2024-02-10T07:45:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:45:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:45:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:45:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:45:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[39923]: osdmap e366: 8 total, 8 up, 8 in 2024-02-10T07:45:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:45:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:17 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:45:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:17 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:45:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:17 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:45:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:17 smithi155 ceph-mon[32688]: osdmap e366: 8 total, 8 up, 8 in 2024-02-10T07:45:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:45:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:17 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:45:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:45:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[29548]: osdmap e367: 8 total, 8 up, 8 in 2024-02-10T07:45:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:45:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:45:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[29548]: pgmap v497: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:45:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[39923]: osdmap e367: 8 total, 8 up, 8 in 2024-02-10T07:45:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:45:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:45:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:18 smithi070 ceph-mon[39923]: pgmap v497: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:45:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:18 smithi155 ceph-mon[32688]: osdmap e367: 8 total, 8 up, 8 in 2024-02-10T07:45:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:45:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:45:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:18 smithi155 ceph-mon[32688]: pgmap v497: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:19.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:45:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:45:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:45:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:19 smithi070 ceph-mon[29548]: osdmap e368: 8 total, 8 up, 8 in 2024-02-10T07:45:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:19 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:45:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:19 smithi070 ceph-mon[39923]: osdmap e368: 8 total, 8 up, 8 in 2024-02-10T07:45:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:19 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-51","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:45:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:19 smithi155 ceph-mon[32688]: osdmap e368: 8 total, 8 up, 8 in 2024-02-10T07:45:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:19 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:20 smithi070 ceph-mon[39923]: pgmap v499: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:20 smithi070 ceph-mon[29548]: pgmap v499: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:20 smithi155 ceph-mon[32688]: pgmap v499: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:21 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:45:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:21 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:45:22.004 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:21 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:45:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:45:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:45:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:23 smithi070 ceph-mon[29548]: pgmap v500: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 0 op/s 2024-02-10T07:45:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:23 smithi070 ceph-mon[39923]: pgmap v500: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 0 op/s 2024-02-10T07:45:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:23 smithi155 ceph-mon[32688]: pgmap v500: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 0 op/s 2024-02-10T07:45:24.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:24 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:45:24.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:24 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:45:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:24 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:45:25.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[39923]: pgmap v501: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:25.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:45:25.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[29548]: pgmap v501: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:25 smithi155 ceph-mon[32688]: pgmap v501: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:45:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:45:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:26.392 INFO:tasks.workunit.client.0.smithi070.stdout: started 0x564b26291ae0 2024-02-10T07:45:26.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26291c10 2024-02-10T07:45:26.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26291d60 2024-02-10T07:45:26.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262920b0 2024-02-10T07:45:26.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292540 2024-02-10T07:45:26.393 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292690 2024-02-10T07:45:26.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26290f30 2024-02-10T07:45:26.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26291060 2024-02-10T07:45:26.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2628dac0 2024-02-10T07:45:26.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262927c0 2024-02-10T07:45:26.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292c70 2024-02-10T07:45:26.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292d80 2024-02-10T07:45:26.394 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293370 2024-02-10T07:45:26.395 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292910 2024-02-10T07:45:26.395 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292a40 2024-02-10T07:45:26.395 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292b50 2024-02-10T07:45:26.395 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262934a0 2024-02-10T07:45:26.395 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292ed0 2024-02-10T07:45:26.395 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292fe0 2024-02-10T07:45:26.396 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293130 2024-02-10T07:45:26.396 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262935f0 2024-02-10T07:45:26.396 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293a80 2024-02-10T07:45:26.396 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293bb0 2024-02-10T07:45:26.396 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262921e0 2024-02-10T07:45:26.396 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292310 2024-02-10T07:45:26.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26292420 2024-02-10T07:45:26.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293ce0 2024-02-10T07:45:26.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262941c0 2024-02-10T07:45:26.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294340 2024-02-10T07:45:26.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294930 2024-02-10T07:45:26.397 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293e10 2024-02-10T07:45:26.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293f40 2024-02-10T07:45:26.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294490 2024-02-10T07:45:26.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262945c0 2024-02-10T07:45:26.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262946f0 2024-02-10T07:45:26.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294820 2024-02-10T07:45:26.398 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294050 2024-02-10T07:45:26.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294a60 2024-02-10T07:45:26.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294bd0 2024-02-10T07:45:26.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262951c0 2024-02-10T07:45:26.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294d20 2024-02-10T07:45:26.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294e70 2024-02-10T07:45:26.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26294fc0 2024-02-10T07:45:26.399 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262952f0 2024-02-10T07:45:26.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262957a0 2024-02-10T07:45:26.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262958b0 2024-02-10T07:45:26.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293720 2024-02-10T07:45:26.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293830 2024-02-10T07:45:26.400 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26293960 2024-02-10T07:45:26.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262959e0 2024-02-10T07:45:26.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26295e70 2024-02-10T07:45:26.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26295fa0 2024-02-10T07:45:26.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26295440 2024-02-10T07:45:26.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26295570 2024-02-10T07:45:26.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296590 2024-02-10T07:45:26.401 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262960f0 2024-02-10T07:45:26.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296200 2024-02-10T07:45:26.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26295680 2024-02-10T07:45:26.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296330 2024-02-10T07:45:26.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296460 2024-02-10T07:45:26.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262966e0 2024-02-10T07:45:26.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296870 2024-02-10T07:45:26.402 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296e60 2024-02-10T07:45:26.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262969c0 2024-02-10T07:45:26.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296af0 2024-02-10T07:45:26.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296c00 2024-02-10T07:45:26.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296d30 2024-02-10T07:45:26.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297410 2024-02-10T07:45:26.403 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26296f70 2024-02-10T07:45:26.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297080 2024-02-10T07:45:26.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262971b0 2024-02-10T07:45:26.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262972e0 2024-02-10T07:45:26.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297a00 2024-02-10T07:45:26.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297560 2024-02-10T07:45:26.404 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297690 2024-02-10T07:45:26.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262977c0 2024-02-10T07:45:26.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262978f0 2024-02-10T07:45:26.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297fd0 2024-02-10T07:45:26.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297b30 2024-02-10T07:45:26.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297c80 2024-02-10T07:45:26.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26297db0 2024-02-10T07:45:26.405 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298100 2024-02-10T07:45:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:27 smithi070 ceph-mon[29548]: pgmap v502: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:27.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:27 smithi070 ceph-mon[39923]: pgmap v502: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:27 smithi155 ceph-mon[32688]: pgmap v502: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:29.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:45:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:45:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:29 smithi155 ceph-mon[32688]: pgmap v503: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:29 smithi070 ceph-mon[39923]: pgmap v503: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:29 smithi070 ceph-mon[29548]: pgmap v503: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:29.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:45:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[29548]: osdmap e369: 8 total, 8 up, 8 in 2024-02-10T07:45:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:45:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[39923]: osdmap e369: 8 total, 8 up, 8 in 2024-02-10T07:45:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:30.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:45:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:30 smithi155 ceph-mon[32688]: osdmap e369: 8 total, 8 up, 8 in 2024-02-10T07:45:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[29548]: pgmap v505: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]': finished 2024-02-10T07:45:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[29548]: osdmap e370: 8 total, 8 up, 8 in 2024-02-10T07:45:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[39923]: pgmap v505: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]': finished 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[39923]: osdmap e370: 8 total, 8 up, 8 in 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:31 smithi155 ceph-mon[32688]: pgmap v505: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:45:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]': finished 2024-02-10T07:45:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:31 smithi155 ceph-mon[32688]: osdmap e370: 8 total, 8 up, 8 in 2024-02-10T07:45:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-51"}]: dispatch 2024-02-10T07:45:32.385 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:45:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:45:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:32 smithi070 ceph-mon[29548]: osdmap e371: 8 total, 8 up, 8 in 2024-02-10T07:45:32.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:32 smithi070 ceph-mon[39923]: osdmap e371: 8 total, 8 up, 8 in 2024-02-10T07:45:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:32 smithi155 ceph-mon[32688]: osdmap e371: 8 total, 8 up, 8 in 2024-02-10T07:45:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[29548]: pgmap v508: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T07:45:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:45:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[29548]: osdmap e372: 8 total, 8 up, 8 in 2024-02-10T07:45:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[39923]: pgmap v508: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T07:45:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:45:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[39923]: osdmap e372: 8 total, 8 up, 8 in 2024-02-10T07:45:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:33 smithi155 ceph-mon[32688]: pgmap v508: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T07:45:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:33 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:45:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:33 smithi155 ceph-mon[32688]: osdmap e372: 8 total, 8 up, 8 in 2024-02-10T07:45:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[39923]: osdmap e373: 8 total, 8 up, 8 in 2024-02-10T07:45:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-53"}]: dispatch 2024-02-10T07:45:34.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-53"}]: dispatch 2024-02-10T07:45:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[29548]: osdmap e373: 8 total, 8 up, 8 in 2024-02-10T07:45:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-53"}]: dispatch 2024-02-10T07:45:34.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:34 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-53"}]: dispatch 2024-02-10T07:45:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:34 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:34 smithi155 ceph-mon[32688]: osdmap e373: 8 total, 8 up, 8 in 2024-02-10T07:45:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:34 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-53"}]: dispatch 2024-02-10T07:45:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:34 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-53"}]: dispatch 2024-02-10T07:45:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:35 smithi070 ceph-mon[29548]: pgmap v511: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:45:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:35 smithi070 ceph-mon[29548]: osdmap e374: 8 total, 8 up, 8 in 2024-02-10T07:45:35.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:35 smithi070 ceph-mon[39923]: pgmap v511: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:45:35.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:35 smithi070 ceph-mon[39923]: osdmap e374: 8 total, 8 up, 8 in 2024-02-10T07:45:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:35 smithi155 ceph-mon[32688]: pgmap v511: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:45:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:35 smithi155 ceph-mon[32688]: osdmap e374: 8 total, 8 up, 8 in 2024-02-10T07:45:36.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:36 smithi070 ceph-mon[29548]: osdmap e375: 8 total, 8 up, 8 in 2024-02-10T07:45:36.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:36.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:36 smithi070 ceph-mon[39923]: osdmap e375: 8 total, 8 up, 8 in 2024-02-10T07:45:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:36.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:36 smithi155 ceph-mon[32688]: osdmap e375: 8 total, 8 up, 8 in 2024-02-10T07:45:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: pgmap v514: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: osdmap e376: 8 total, 8 up, 8 in 2024-02-10T07:45:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-55"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-55"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: pgmap v514: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: osdmap e376: 8 total, 8 up, 8 in 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-55"}]: dispatch 2024-02-10T07:45:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:37 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-55"}]: dispatch 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: pgmap v514: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: osdmap e376: 8 total, 8 up, 8 in 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-55"}]: dispatch 2024-02-10T07:45:37.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:37 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-55"}]: dispatch 2024-02-10T07:45:38.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:38 smithi070 ceph-mon[29548]: osdmap e377: 8 total, 8 up, 8 in 2024-02-10T07:45:38.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:38 smithi070 ceph-mon[39923]: osdmap e377: 8 total, 8 up, 8 in 2024-02-10T07:45:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:38 smithi155 ceph-mon[32688]: osdmap e377: 8 total, 8 up, 8 in 2024-02-10T07:45:39.436 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:45:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:45:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[29548]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[29548]: osdmap e378: 8 total, 8 up, 8 in 2024-02-10T07:45:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[39923]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[39923]: osdmap e378: 8 total, 8 up, 8 in 2024-02-10T07:45:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:39.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:39 smithi155 ceph-mon[32688]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:39 smithi155 ceph-mon[32688]: osdmap e378: 8 total, 8 up, 8 in 2024-02-10T07:45:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:40.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:40.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:40.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: osdmap e379: 8 total, 8 up, 8 in 2024-02-10T07:45:40.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:40.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-57"}]: dispatch 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-57"}]: dispatch 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: osdmap e379: 8 total, 8 up, 8 in 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:40.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:40.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-57"}]: dispatch 2024-02-10T07:45:40.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-57"}]: dispatch 2024-02-10T07:45:40.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:40.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: osdmap e379: 8 total, 8 up, 8 in 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-57"}]: dispatch 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-57"}]: dispatch 2024-02-10T07:45:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:45:40.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:41 smithi070 ceph-mon[29548]: pgmap v520: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:45:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:41 smithi070 ceph-mon[29548]: osdmap e380: 8 total, 8 up, 8 in 2024-02-10T07:45:41.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:41 smithi070 ceph-mon[39923]: pgmap v520: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:45:41.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:41 smithi070 ceph-mon[39923]: osdmap e380: 8 total, 8 up, 8 in 2024-02-10T07:45:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:41 smithi155 ceph-mon[32688]: pgmap v520: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:45:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:41 smithi155 ceph-mon[32688]: osdmap e380: 8 total, 8 up, 8 in 2024-02-10T07:45:42.450 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:45:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[29548]: osdmap e381: 8 total, 8 up, 8 in 2024-02-10T07:45:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:42.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:42.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[39923]: osdmap e381: 8 total, 8 up, 8 in 2024-02-10T07:45:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:42 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:42 smithi155 ceph-mon[32688]: osdmap e381: 8 total, 8 up, 8 in 2024-02-10T07:45:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:42 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:45:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: pgmap v523: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:45:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: osdmap e382: 8 total, 8 up, 8 in 2024-02-10T07:45:43.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-59"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-59"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: pgmap v523: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: osdmap e382: 8 total, 8 up, 8 in 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-59"}]: dispatch 2024-02-10T07:45:43.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-59"}]: dispatch 2024-02-10T07:45:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: pgmap v523: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:45:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: osdmap e382: 8 total, 8 up, 8 in 2024-02-10T07:45:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-59"}]: dispatch 2024-02-10T07:45:43.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-59"}]: dispatch 2024-02-10T07:45:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:44 smithi070 ceph-mon[29548]: osdmap e383: 8 total, 8 up, 8 in 2024-02-10T07:45:44.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:44 smithi070 ceph-mon[39923]: osdmap e383: 8 total, 8 up, 8 in 2024-02-10T07:45:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:44 smithi155 ceph-mon[32688]: osdmap e383: 8 total, 8 up, 8 in 2024-02-10T07:45:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[29548]: pgmap v526: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:45:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[29548]: osdmap e384: 8 total, 8 up, 8 in 2024-02-10T07:45:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[39923]: pgmap v526: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:45:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[39923]: osdmap e384: 8 total, 8 up, 8 in 2024-02-10T07:45:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:45 smithi155 ceph-mon[32688]: pgmap v526: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:45:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:45 smithi155 ceph-mon[32688]: osdmap e384: 8 total, 8 up, 8 in 2024-02-10T07:45:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[29548]: osdmap e385: 8 total, 8 up, 8 in 2024-02-10T07:45:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[39923]: osdmap e385: 8 total, 8 up, 8 in 2024-02-10T07:45:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:46 smithi155 ceph-mon[32688]: osdmap e385: 8 total, 8 up, 8 in 2024-02-10T07:45:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[29548]: pgmap v529: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:45:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:45:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[29548]: osdmap e386: 8 total, 8 up, 8 in 2024-02-10T07:45:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-61"}]: dispatch 2024-02-10T07:45:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-61"}]: dispatch 2024-02-10T07:45:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[39923]: pgmap v529: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:45:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:45:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[39923]: osdmap e386: 8 total, 8 up, 8 in 2024-02-10T07:45:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-61"}]: dispatch 2024-02-10T07:45:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-61"}]: dispatch 2024-02-10T07:45:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:47 smithi155 ceph-mon[32688]: pgmap v529: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T07:45:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:45:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:47 smithi155 ceph-mon[32688]: osdmap e386: 8 total, 8 up, 8 in 2024-02-10T07:45:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:45:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-61"}]: dispatch 2024-02-10T07:45:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-61"}]: dispatch 2024-02-10T07:45:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:48 smithi070 ceph-mon[29548]: osdmap e387: 8 total, 8 up, 8 in 2024-02-10T07:45:48.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:48 smithi070 ceph-mon[39923]: osdmap e387: 8 total, 8 up, 8 in 2024-02-10T07:45:48.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:48 smithi155 ceph-mon[32688]: osdmap e387: 8 total, 8 up, 8 in 2024-02-10T07:45:49.501 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:45:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:45:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:49 smithi155 ceph-mon[32688]: pgmap v532: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:49 smithi155 ceph-mon[32688]: osdmap e388: 8 total, 8 up, 8 in 2024-02-10T07:45:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[29548]: pgmap v532: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[29548]: osdmap e388: 8 total, 8 up, 8 in 2024-02-10T07:45:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[39923]: pgmap v532: 324 pgs: 324 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:45:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[39923]: osdmap e388: 8 total, 8 up, 8 in 2024-02-10T07:45:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:45:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[29548]: osdmap e389: 8 total, 8 up, 8 in 2024-02-10T07:45:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[39923]: osdmap e389: 8 total, 8 up, 8 in 2024-02-10T07:45:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:45:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:45:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:50 smithi155 ceph-mon[32688]: osdmap e389: 8 total, 8 up, 8 in 2024-02-10T07:45:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:45:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:45:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:51 smithi070 ceph-mon[29548]: pgmap v535: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:45:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:51 smithi070 ceph-mon[29548]: osdmap e390: 8 total, 8 up, 8 in 2024-02-10T07:45:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:51 smithi070 ceph-mon[39923]: pgmap v535: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:45:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:51 smithi070 ceph-mon[39923]: osdmap e390: 8 total, 8 up, 8 in 2024-02-10T07:45:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:51 smithi155 ceph-mon[32688]: pgmap v535: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:45:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:51 smithi155 ceph-mon[32688]: osdmap e390: 8 total, 8 up, 8 in 2024-02-10T07:45:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:45:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:45:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:52 smithi070 ceph-mon[29548]: osdmap e391: 8 total, 8 up, 8 in 2024-02-10T07:45:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:52 smithi070 ceph-mon[39923]: osdmap e391: 8 total, 8 up, 8 in 2024-02-10T07:45:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:52 smithi155 ceph-mon[32688]: osdmap e391: 8 total, 8 up, 8 in 2024-02-10T07:45:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:53 smithi070 ceph-mon[29548]: pgmap v538: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:54.062 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:53 smithi070 ceph-mon[29548]: osdmap e392: 8 total, 8 up, 8 in 2024-02-10T07:45:54.062 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:53 smithi070 ceph-mon[39923]: pgmap v538: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:54.062 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:53 smithi070 ceph-mon[39923]: osdmap e392: 8 total, 8 up, 8 in 2024-02-10T07:45:54.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:53 smithi155 ceph-mon[32688]: pgmap v538: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:45:54.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:53 smithi155 ceph-mon[32688]: osdmap e392: 8 total, 8 up, 8 in 2024-02-10T07:45:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:54 smithi070 ceph-mon[29548]: osdmap e393: 8 total, 8 up, 8 in 2024-02-10T07:45:54.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:54 smithi070 ceph-mon[39923]: osdmap e393: 8 total, 8 up, 8 in 2024-02-10T07:45:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:54 smithi155 ceph-mon[32688]: osdmap e393: 8 total, 8 up, 8 in 2024-02-10T07:45:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:55 smithi070 ceph-mon[29548]: pgmap v541: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-10T07:45:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:55 smithi070 ceph-mon[29548]: osdmap e394: 8 total, 8 up, 8 in 2024-02-10T07:45:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:55 smithi070 ceph-mon[39923]: pgmap v541: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-10T07:45:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:55 smithi070 ceph-mon[39923]: osdmap e394: 8 total, 8 up, 8 in 2024-02-10T07:45:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:55 smithi155 ceph-mon[32688]: pgmap v541: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-10T07:45:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:55 smithi155 ceph-mon[32688]: osdmap e394: 8 total, 8 up, 8 in 2024-02-10T07:45:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:57 smithi070 ceph-mon[29548]: pgmap v543: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-10T07:45:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:57 smithi070 ceph-mon[39923]: pgmap v543: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-10T07:45:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:57 smithi155 ceph-mon[32688]: pgmap v543: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-10T07:45:59.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:45:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:45:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:45:59 smithi070 ceph-mon[29548]: pgmap v544: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-10T07:45:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:45:59 smithi070 ceph-mon[39923]: pgmap v544: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-10T07:46:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:45:59 smithi155 ceph-mon[32688]: pgmap v544: 356 pgs: 356 active+clean; 8.4 MiB data, 674 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-10T07:46:00.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:01 smithi070 ceph-mon[29548]: pgmap v545: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:46:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:01 smithi070 ceph-mon[39923]: pgmap v545: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:46:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:01 smithi155 ceph-mon[32688]: pgmap v545: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:46:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:46:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:46:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:46:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:03 smithi070 ceph-mon[29548]: pgmap v546: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:46:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:03 smithi070 ceph-mon[29548]: osdmap e395: 8 total, 8 up, 8 in 2024-02-10T07:46:03.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:03 smithi070 ceph-mon[39923]: pgmap v546: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:46:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:03 smithi070 ceph-mon[39923]: osdmap e395: 8 total, 8 up, 8 in 2024-02-10T07:46:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:03 smithi155 ceph-mon[32688]: pgmap v546: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:46:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:03 smithi155 ceph-mon[32688]: osdmap e395: 8 total, 8 up, 8 in 2024-02-10T07:46:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:05 smithi070 ceph-mon[29548]: pgmap v548: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:05 smithi070 ceph-mon[39923]: pgmap v548: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:05 smithi155 ceph-mon[32688]: pgmap v548: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:06 smithi070 ceph-mon[29548]: osdmap e396: 8 total, 8 up, 8 in 2024-02-10T07:46:06.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:06 smithi070 ceph-mon[39923]: osdmap e396: 8 total, 8 up, 8 in 2024-02-10T07:46:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:06 smithi155 ceph-mon[32688]: osdmap e396: 8 total, 8 up, 8 in 2024-02-10T07:46:07.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:07 smithi155 ceph-mon[32688]: pgmap v550: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:07 smithi070 ceph-mon[29548]: pgmap v550: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:07.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:07 smithi070 ceph-mon[39923]: pgmap v550: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:09.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:46:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:46:09.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:09 smithi155 ceph-mon[32688]: pgmap v551: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:46:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:09 smithi070 ceph-mon[29548]: pgmap v551: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:46:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:09 smithi070 ceph-mon[39923]: pgmap v551: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:46:10.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:10.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:10.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:11.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:11 smithi070 ceph-mon[39923]: pgmap v552: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:11 smithi070 ceph-mon[29548]: pgmap v552: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:11 smithi155 ceph-mon[32688]: pgmap v552: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:46:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:46:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:46:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:13 smithi070 ceph-mon[39923]: pgmap v553: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:13 smithi070 ceph-mon[39923]: osdmap e397: 8 total, 8 up, 8 in 2024-02-10T07:46:13.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:13 smithi070 ceph-mon[29548]: pgmap v553: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:13 smithi070 ceph-mon[29548]: osdmap e397: 8 total, 8 up, 8 in 2024-02-10T07:46:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:13 smithi155 ceph-mon[32688]: pgmap v553: 356 pgs: 356 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:13 smithi155 ceph-mon[32688]: osdmap e397: 8 total, 8 up, 8 in 2024-02-10T07:46:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:15 smithi070 ceph-mon[29548]: pgmap v555: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:15 smithi070 ceph-mon[39923]: pgmap v555: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:15 smithi155 ceph-mon[32688]: pgmap v555: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:17 smithi070 ceph-mon[29548]: pgmap v556: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:17 smithi070 ceph-mon[29548]: osdmap e398: 8 total, 8 up, 8 in 2024-02-10T07:46:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:17 smithi070 ceph-mon[39923]: pgmap v556: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:17 smithi070 ceph-mon[39923]: osdmap e398: 8 total, 8 up, 8 in 2024-02-10T07:46:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:17 smithi155 ceph-mon[32688]: pgmap v556: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:17 smithi155 ceph-mon[32688]: osdmap e398: 8 total, 8 up, 8 in 2024-02-10T07:46:19.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:19 smithi155 ceph-mon[32688]: pgmap v558: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:46:19.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:46:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:46:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:19 smithi070 ceph-mon[29548]: pgmap v558: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:46:19.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:19 smithi070 ceph-mon[39923]: pgmap v558: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:46:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:20.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:20.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:21 smithi070 ceph-mon[29548]: pgmap v559: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:46:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:21 smithi070 ceph-mon[39923]: pgmap v559: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:46:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:21 smithi155 ceph-mon[32688]: pgmap v559: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:46:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:46:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:46:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:46:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:23 smithi070 ceph-mon[29548]: pgmap v560: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:23 smithi070 ceph-mon[29548]: osdmap e399: 8 total, 8 up, 8 in 2024-02-10T07:46:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:23 smithi070 ceph-mon[39923]: pgmap v560: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:23 smithi070 ceph-mon[39923]: osdmap e399: 8 total, 8 up, 8 in 2024-02-10T07:46:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:23 smithi155 ceph-mon[32688]: pgmap v560: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:23 smithi155 ceph-mon[32688]: osdmap e399: 8 total, 8 up, 8 in 2024-02-10T07:46:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[29548]: pgmap v562: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:46:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:46:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[39923]: pgmap v562: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:46:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:25 smithi155 ceph-mon[32688]: pgmap v562: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:46:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:46:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:46:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:46:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:46:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[29548]: pgmap v563: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-63"}]: dispatch 2024-02-10T07:46:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-63"}]: dispatch 2024-02-10T07:46:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[39923]: pgmap v563: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-63"}]: dispatch 2024-02-10T07:46:27.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:27 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-63"}]: dispatch 2024-02-10T07:46:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:27 smithi155 ceph-mon[32688]: pgmap v563: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:27 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-63"}]: dispatch 2024-02-10T07:46:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:27 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-63"}]: dispatch 2024-02-10T07:46:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:28 smithi070 ceph-mon[29548]: osdmap e400: 8 total, 8 up, 8 in 2024-02-10T07:46:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:28 smithi070 ceph-mon[39923]: osdmap e400: 8 total, 8 up, 8 in 2024-02-10T07:46:28.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:28 smithi155 ceph-mon[32688]: osdmap e400: 8 total, 8 up, 8 in 2024-02-10T07:46:29.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:46:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:46:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:29 smithi155 ceph-mon[32688]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:29 smithi155 ceph-mon[32688]: osdmap e401: 8 total, 8 up, 8 in 2024-02-10T07:46:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:29.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[29548]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[29548]: osdmap e401: 8 total, 8 up, 8 in 2024-02-10T07:46:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[39923]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[39923]: osdmap e401: 8 total, 8 up, 8 in 2024-02-10T07:46:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:29.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[39923]: osdmap e402: 8 total, 8 up, 8 in 2024-02-10T07:46:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[29548]: osdmap e402: 8 total, 8 up, 8 in 2024-02-10T07:46:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:30.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:30 smithi155 ceph-mon[32688]: osdmap e402: 8 total, 8 up, 8 in 2024-02-10T07:46:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:31 smithi070 ceph-mon[29548]: pgmap v568: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:46:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:31 smithi070 ceph-mon[29548]: osdmap e403: 8 total, 8 up, 8 in 2024-02-10T07:46:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:31 smithi070 ceph-mon[39923]: pgmap v568: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:46:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:31 smithi070 ceph-mon[39923]: osdmap e403: 8 total, 8 up, 8 in 2024-02-10T07:46:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:31 smithi155 ceph-mon[32688]: pgmap v568: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:46:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:31 smithi155 ceph-mon[32688]: osdmap e403: 8 total, 8 up, 8 in 2024-02-10T07:46:32.411 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:46:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:46:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:46:32.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:32 smithi070 ceph-mon[39923]: osdmap e404: 8 total, 8 up, 8 in 2024-02-10T07:46:32.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:32 smithi070 ceph-mon[29548]: osdmap e404: 8 total, 8 up, 8 in 2024-02-10T07:46:32.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:32 smithi155 ceph-mon[32688]: osdmap e404: 8 total, 8 up, 8 in 2024-02-10T07:46:33.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:33 smithi070 ceph-mon[39923]: pgmap v571: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:33.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:33 smithi070 ceph-mon[39923]: osdmap e405: 8 total, 8 up, 8 in 2024-02-10T07:46:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:33 smithi070 ceph-mon[29548]: pgmap v571: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:33 smithi070 ceph-mon[29548]: osdmap e405: 8 total, 8 up, 8 in 2024-02-10T07:46:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:33 smithi155 ceph-mon[32688]: pgmap v571: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:46:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:33 smithi155 ceph-mon[32688]: osdmap e405: 8 total, 8 up, 8 in 2024-02-10T07:46:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:34 smithi070 ceph-mon[29548]: osdmap e406: 8 total, 8 up, 8 in 2024-02-10T07:46:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:34 smithi070 ceph-mon[39923]: osdmap e406: 8 total, 8 up, 8 in 2024-02-10T07:46:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:34 smithi155 ceph-mon[32688]: osdmap e406: 8 total, 8 up, 8 in 2024-02-10T07:46:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:35 smithi070 ceph-mon[29548]: pgmap v574: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-02-10T07:46:35.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:35 smithi070 ceph-mon[39923]: pgmap v574: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-02-10T07:46:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:35 smithi155 ceph-mon[32688]: pgmap v574: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-02-10T07:46:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:37 smithi070 ceph-mon[29548]: pgmap v575: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-02-10T07:46:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:37 smithi070 ceph-mon[39923]: pgmap v575: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-02-10T07:46:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:37 smithi155 ceph-mon[32688]: pgmap v575: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-02-10T07:46:39.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:46:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:46:39.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:39 smithi155 ceph-mon[32688]: pgmap v576: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-10T07:46:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:39 smithi070 ceph-mon[29548]: pgmap v576: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-10T07:46:39.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:39 smithi070 ceph-mon[39923]: pgmap v576: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-10T07:46:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:40.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:40.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:41.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:41 smithi070 ceph-mon[29548]: pgmap v577: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T07:46:41.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:41 smithi070 ceph-mon[39923]: pgmap v577: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T07:46:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:41 smithi155 ceph-mon[32688]: pgmap v577: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T07:46:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:46:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:46:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:46:43.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:43 smithi070 ceph-mon[29548]: pgmap v578: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-10T07:46:43.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:43 smithi070 ceph-mon[29548]: osdmap e407: 8 total, 8 up, 8 in 2024-02-10T07:46:43.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:43 smithi070 ceph-mon[39923]: pgmap v578: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-10T07:46:43.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:43 smithi070 ceph-mon[39923]: osdmap e407: 8 total, 8 up, 8 in 2024-02-10T07:46:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:43 smithi155 ceph-mon[32688]: pgmap v578: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-10T07:46:43.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:43 smithi155 ceph-mon[32688]: osdmap e407: 8 total, 8 up, 8 in 2024-02-10T07:46:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-65"}]: dispatch 2024-02-10T07:46:44.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-65"}]: dispatch 2024-02-10T07:46:44.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:44.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:44.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-65"}]: dispatch 2024-02-10T07:46:44.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-65"}]: dispatch 2024-02-10T07:46:44.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-65"}]: dispatch 2024-02-10T07:46:44.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-65"}]: dispatch 2024-02-10T07:46:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:45 smithi070 ceph-mon[29548]: pgmap v580: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:45 smithi070 ceph-mon[29548]: osdmap e408: 8 total, 8 up, 8 in 2024-02-10T07:46:45.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:45 smithi070 ceph-mon[39923]: pgmap v580: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:45.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:45 smithi070 ceph-mon[39923]: osdmap e408: 8 total, 8 up, 8 in 2024-02-10T07:46:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:45 smithi155 ceph-mon[32688]: pgmap v580: 356 pgs: 356 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:46:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:45 smithi155 ceph-mon[32688]: osdmap e408: 8 total, 8 up, 8 in 2024-02-10T07:46:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:46 smithi070 ceph-mon[29548]: osdmap e409: 8 total, 8 up, 8 in 2024-02-10T07:46:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:46 smithi070 ceph-mon[39923]: osdmap e409: 8 total, 8 up, 8 in 2024-02-10T07:46:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:46 smithi155 ceph-mon[32688]: osdmap e409: 8 total, 8 up, 8 in 2024-02-10T07:46:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:47.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[39923]: pgmap v583: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:47.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:46:47.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[39923]: osdmap e410: 8 total, 8 up, 8 in 2024-02-10T07:46:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[29548]: pgmap v583: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:46:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[29548]: osdmap e410: 8 total, 8 up, 8 in 2024-02-10T07:46:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:47 smithi155 ceph-mon[32688]: pgmap v583: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:47 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:46:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:47 smithi155 ceph-mon[32688]: osdmap e410: 8 total, 8 up, 8 in 2024-02-10T07:46:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:48.408 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262985e0 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298760 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26295b10 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26295c40 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298d30 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298890 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298230 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262996b0 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299210 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298360 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298490 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299340 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299470 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262997e0 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299c90 2024-02-10T07:46:48.409 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299dc0 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298e60 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26298f90 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262995a0 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262990c0 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629a390 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299ef0 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629a040 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629a190 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629a4c0 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629a950 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629aa60 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299910 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299a40 2024-02-10T07:46:48.410 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b050 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629abb0 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629ace0 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629ae10 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629af40 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b620 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b180 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b2d0 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b400 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b750 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629bbe0 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629bd10 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629a5f0 2024-02-10T07:46:48.411 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629a720 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c2e0 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629be40 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629bf70 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c0a0 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c1d0 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c8b0 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c410 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c540 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c670 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c7a0 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629ce80 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629c9e0 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629cb30 2024-02-10T07:46:48.412 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629cc40 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629cd70 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629d450 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629cfb0 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629d100 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629d230 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629d580 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629da10 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629db40 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629d340 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629dc70 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629dde0 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b26299b50 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629df30 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e080 2024-02-10T07:46:48.413 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b880 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629b9d0 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e670 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e1d0 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e300 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e430 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e560 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629ec40 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e7a0 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629e8d0 2024-02-10T07:46:48.414 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629ea00 2024-02-10T07:46:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[29548]: osdmap e411: 8 total, 8 up, 8 in 2024-02-10T07:46:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-67"}]: dispatch 2024-02-10T07:46:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-67"}]: dispatch 2024-02-10T07:46:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[39923]: osdmap e411: 8 total, 8 up, 8 in 2024-02-10T07:46:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-67"}]: dispatch 2024-02-10T07:46:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-67"}]: dispatch 2024-02-10T07:46:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:48 smithi155 ceph-mon[32688]: osdmap e411: 8 total, 8 up, 8 in 2024-02-10T07:46:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-67"}]: dispatch 2024-02-10T07:46:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-67"}]: dispatch 2024-02-10T07:46:49.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:49 smithi155 ceph-mon[32688]: pgmap v586: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:49.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:49 smithi155 ceph-mon[32688]: osdmap e412: 8 total, 8 up, 8 in 2024-02-10T07:46:49.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:46:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:46:49.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:49 smithi070 ceph-mon[29548]: pgmap v586: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:49.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:49 smithi070 ceph-mon[29548]: osdmap e412: 8 total, 8 up, 8 in 2024-02-10T07:46:49.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:49 smithi070 ceph-mon[39923]: pgmap v586: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:46:49.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:49 smithi070 ceph-mon[39923]: osdmap e412: 8 total, 8 up, 8 in 2024-02-10T07:46:50.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:50.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[29548]: osdmap e413: 8 total, 8 up, 8 in 2024-02-10T07:46:50.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:50.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:50.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:50.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[39923]: osdmap e413: 8 total, 8 up, 8 in 2024-02-10T07:46:50.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:50.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:50.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:46:50.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:50 smithi155 ceph-mon[32688]: osdmap e413: 8 total, 8 up, 8 in 2024-02-10T07:46:50.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:50.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:51.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: pgmap v589: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:46:51.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:51.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: osdmap e414: 8 total, 8 up, 8 in 2024-02-10T07:46:51.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:51.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-69"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-69"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: pgmap v589: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: osdmap e414: 8 total, 8 up, 8 in 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-69"}]: dispatch 2024-02-10T07:46:51.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-69"}]: dispatch 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: pgmap v589: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: osdmap e414: 8 total, 8 up, 8 in 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-69"}]: dispatch 2024-02-10T07:46:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-69"}]: dispatch 2024-02-10T07:46:52.425 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:46:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:46:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:46:52.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:52 smithi070 ceph-mon[29548]: osdmap e415: 8 total, 8 up, 8 in 2024-02-10T07:46:52.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:52 smithi070 ceph-mon[39923]: osdmap e415: 8 total, 8 up, 8 in 2024-02-10T07:46:52.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:52 smithi155 ceph-mon[32688]: osdmap e415: 8 total, 8 up, 8 in 2024-02-10T07:46:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[29548]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:46:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:46:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[29548]: osdmap e416: 8 total, 8 up, 8 in 2024-02-10T07:46:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:53.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[39923]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:46:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:46:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[39923]: osdmap e416: 8 total, 8 up, 8 in 2024-02-10T07:46:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:53 smithi155 ceph-mon[32688]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:46:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:53 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:46:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:53 smithi155 ceph-mon[32688]: osdmap e416: 8 total, 8 up, 8 in 2024-02-10T07:46:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:46:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[29548]: osdmap e417: 8 total, 8 up, 8 in 2024-02-10T07:46:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[39923]: osdmap e417: 8 total, 8 up, 8 in 2024-02-10T07:46:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:46:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:54 smithi155 ceph-mon[32688]: osdmap e417: 8 total, 8 up, 8 in 2024-02-10T07:46:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:46:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:54.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:46:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[29548]: pgmap v595: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:46:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:46:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[29548]: osdmap e418: 8 total, 8 up, 8 in 2024-02-10T07:46:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[39923]: pgmap v595: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:46:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:46:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[39923]: osdmap e418: 8 total, 8 up, 8 in 2024-02-10T07:46:55.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:55 smithi155 ceph-mon[32688]: pgmap v595: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:46:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:46:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:55 smithi155 ceph-mon[32688]: osdmap e418: 8 total, 8 up, 8 in 2024-02-10T07:46:55.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:46:56.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:56 smithi070 ceph-mon[29548]: osdmap e419: 8 total, 8 up, 8 in 2024-02-10T07:46:56.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:56 smithi070 ceph-mon[39923]: osdmap e419: 8 total, 8 up, 8 in 2024-02-10T07:46:56.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:56 smithi155 ceph-mon[32688]: osdmap e419: 8 total, 8 up, 8 in 2024-02-10T07:46:57.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:57 smithi070 ceph-mon[29548]: pgmap v598: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:46:57.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:57 smithi070 ceph-mon[39923]: pgmap v598: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:46:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:57 smithi155 ceph-mon[32688]: pgmap v598: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:46:59.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:46:59 smithi155 ceph-mon[32688]: pgmap v599: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 559 B/s wr, 1 op/s 2024-02-10T07:46:59.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:46:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:46:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:46:59 smithi070 ceph-mon[29548]: pgmap v599: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 559 B/s wr, 1 op/s 2024-02-10T07:46:59.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:46:59 smithi070 ceph-mon[39923]: pgmap v599: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 559 B/s wr, 1 op/s 2024-02-10T07:47:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:00.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:00.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:01 smithi070 ceph-mon[29548]: pgmap v600: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-10T07:47:01.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:01 smithi070 ceph-mon[39923]: pgmap v600: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-10T07:47:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:01 smithi155 ceph-mon[32688]: pgmap v600: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-02-10T07:47:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:47:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:47:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:47:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:03 smithi070 ceph-mon[29548]: pgmap v601: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-10T07:47:03.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:03 smithi070 ceph-mon[39923]: pgmap v601: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-10T07:47:03.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:03 smithi155 ceph-mon[32688]: pgmap v601: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-10T07:47:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:05 smithi070 ceph-mon[29548]: pgmap v602: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-10T07:47:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:05 smithi070 ceph-mon[39923]: pgmap v602: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-10T07:47:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:05 smithi155 ceph-mon[32688]: pgmap v602: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-10T07:47:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:07 smithi070 ceph-mon[29548]: pgmap v603: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:47:07.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:07 smithi070 ceph-mon[39923]: pgmap v603: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:47:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:07 smithi155 ceph-mon[32688]: pgmap v603: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:47:09.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:09 smithi155 ceph-mon[32688]: pgmap v604: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-10T07:47:09.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:47:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:47:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:09 smithi070 ceph-mon[29548]: pgmap v604: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-10T07:47:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:09 smithi070 ceph-mon[39923]: pgmap v604: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-10T07:47:10.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:10.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:11.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:11 smithi070 ceph-mon[39923]: pgmap v605: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:47:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:11 smithi070 ceph-mon[29548]: pgmap v605: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:47:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:11 smithi155 ceph-mon[32688]: pgmap v605: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:47:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:47:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:47:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:47:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:13 smithi070 ceph-mon[39923]: pgmap v606: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:13.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:13 smithi070 ceph-mon[29548]: pgmap v606: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:13 smithi155 ceph-mon[32688]: pgmap v606: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:15 smithi070 ceph-mon[39923]: pgmap v607: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:15 smithi070 ceph-mon[29548]: pgmap v607: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:15 smithi155 ceph-mon[32688]: pgmap v607: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-71"}]: dispatch 2024-02-10T07:47:16.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-71"}]: dispatch 2024-02-10T07:47:16.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:16.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:16.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-71"}]: dispatch 2024-02-10T07:47:16.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:16 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-71"}]: dispatch 2024-02-10T07:47:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:16 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-71"}]: dispatch 2024-02-10T07:47:16.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:16 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-71"}]: dispatch 2024-02-10T07:47:17.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:17 smithi070 ceph-mon[39923]: pgmap v608: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:17.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:17 smithi070 ceph-mon[39923]: osdmap e420: 8 total, 8 up, 8 in 2024-02-10T07:47:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:17 smithi070 ceph-mon[29548]: pgmap v608: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:17 smithi070 ceph-mon[29548]: osdmap e420: 8 total, 8 up, 8 in 2024-02-10T07:47:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:17 smithi155 ceph-mon[32688]: pgmap v608: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:47:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:17 smithi155 ceph-mon[32688]: osdmap e420: 8 total, 8 up, 8 in 2024-02-10T07:47:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:18 smithi155 ceph-mon[32688]: osdmap e421: 8 total, 8 up, 8 in 2024-02-10T07:47:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:18 smithi070 ceph-mon[29548]: osdmap e421: 8 total, 8 up, 8 in 2024-02-10T07:47:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:18 smithi070 ceph-mon[39923]: osdmap e421: 8 total, 8 up, 8 in 2024-02-10T07:47:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:19.501 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:47:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:47:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:19 smithi155 ceph-mon[32688]: pgmap v611: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:19 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:19 smithi155 ceph-mon[32688]: osdmap e422: 8 total, 8 up, 8 in 2024-02-10T07:47:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[29548]: pgmap v611: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[29548]: osdmap e422: 8 total, 8 up, 8 in 2024-02-10T07:47:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[39923]: pgmap v611: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[39923]: osdmap e422: 8 total, 8 up, 8 in 2024-02-10T07:47:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:20 smithi070 ceph-mon[29548]: osdmap e423: 8 total, 8 up, 8 in 2024-02-10T07:47:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:20 smithi070 ceph-mon[39923]: osdmap e423: 8 total, 8 up, 8 in 2024-02-10T07:47:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:20 smithi155 ceph-mon[32688]: osdmap e423: 8 total, 8 up, 8 in 2024-02-10T07:47:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:21 smithi070 ceph-mon[39923]: pgmap v614: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:21 smithi070 ceph-mon[39923]: osdmap e424: 8 total, 8 up, 8 in 2024-02-10T07:47:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:21 smithi070 ceph-mon[29548]: pgmap v614: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:21 smithi070 ceph-mon[29548]: osdmap e424: 8 total, 8 up, 8 in 2024-02-10T07:47:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:21 smithi155 ceph-mon[32688]: pgmap v614: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:21 smithi155 ceph-mon[32688]: osdmap e424: 8 total, 8 up, 8 in 2024-02-10T07:47:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:47:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:47:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:47:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:23 smithi070 ceph-mon[39923]: pgmap v616: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-10T07:47:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:23 smithi070 ceph-mon[29548]: pgmap v616: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-10T07:47:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:23 smithi155 ceph-mon[32688]: pgmap v616: 356 pgs: 356 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-10T07:47:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:25 smithi070 ceph-mon[39923]: pgmap v617: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T07:47:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:47:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:25 smithi070 ceph-mon[29548]: pgmap v617: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T07:47:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:47:26.004 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:25 smithi155 ceph-mon[32688]: pgmap v617: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-10T07:47:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:47:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:47:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:47:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:47:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:47:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:47:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:47:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:47:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:47:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:47:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:27 smithi070 ceph-mon[29548]: pgmap v618: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 965 B/s wr, 4 op/s 2024-02-10T07:47:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:27 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:27 smithi070 ceph-mon[39923]: pgmap v618: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 965 B/s wr, 4 op/s 2024-02-10T07:47:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:27 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:27 smithi155 ceph-mon[32688]: pgmap v618: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 965 B/s wr, 4 op/s 2024-02-10T07:47:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:27 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:29.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:47:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:47:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:29 smithi070 ceph-mon[39923]: pgmap v619: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 243 B/s wr, 2 op/s 2024-02-10T07:47:29.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:29 smithi070 ceph-mon[29548]: pgmap v619: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 243 B/s wr, 2 op/s 2024-02-10T07:47:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:29 smithi155 ceph-mon[32688]: pgmap v619: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 243 B/s wr, 2 op/s 2024-02-10T07:47:30.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[39923]: pgmap v620: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-73"}]: dispatch 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-73"}]: dispatch 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[29548]: pgmap v620: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-73"}]: dispatch 2024-02-10T07:47:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-73"}]: dispatch 2024-02-10T07:47:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:31 smithi155 ceph-mon[32688]: pgmap v620: 356 pgs: 356 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-10T07:47:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-73"}]: dispatch 2024-02-10T07:47:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-73"}]: dispatch 2024-02-10T07:47:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:47:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:47:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:47:32.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:32 smithi070 ceph-mon[29548]: osdmap e425: 8 total, 8 up, 8 in 2024-02-10T07:47:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:32 smithi070 ceph-mon[39923]: osdmap e425: 8 total, 8 up, 8 in 2024-02-10T07:47:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:32 smithi155 ceph-mon[32688]: osdmap e425: 8 total, 8 up, 8 in 2024-02-10T07:47:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[29548]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:47:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[29548]: osdmap e426: 8 total, 8 up, 8 in 2024-02-10T07:47:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[39923]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:47:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[39923]: osdmap e426: 8 total, 8 up, 8 in 2024-02-10T07:47:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:33 smithi155 ceph-mon[32688]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:47:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:33 smithi155 ceph-mon[32688]: osdmap e426: 8 total, 8 up, 8 in 2024-02-10T07:47:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:34 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:34 smithi070 ceph-mon[29548]: osdmap e427: 8 total, 8 up, 8 in 2024-02-10T07:47:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:34 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:34 smithi070 ceph-mon[39923]: osdmap e427: 8 total, 8 up, 8 in 2024-02-10T07:47:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:34 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:34 smithi155 ceph-mon[32688]: osdmap e427: 8 total, 8 up, 8 in 2024-02-10T07:47:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:35 smithi070 ceph-mon[29548]: pgmap v625: 356 pgs: 13 creating+activating, 343 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:47:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:35 smithi070 ceph-mon[29548]: osdmap e428: 8 total, 8 up, 8 in 2024-02-10T07:47:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:35 smithi070 ceph-mon[39923]: pgmap v625: 356 pgs: 13 creating+activating, 343 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:47:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:35 smithi070 ceph-mon[39923]: osdmap e428: 8 total, 8 up, 8 in 2024-02-10T07:47:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:35 smithi155 ceph-mon[32688]: pgmap v625: 356 pgs: 13 creating+activating, 343 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:47:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:35 smithi155 ceph-mon[32688]: osdmap e428: 8 total, 8 up, 8 in 2024-02-10T07:47:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[29548]: osdmap e429: 8 total, 8 up, 8 in 2024-02-10T07:47:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-75"}]: dispatch 2024-02-10T07:47:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-75"}]: dispatch 2024-02-10T07:47:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[39923]: osdmap e429: 8 total, 8 up, 8 in 2024-02-10T07:47:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-75"}]: dispatch 2024-02-10T07:47:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-75"}]: dispatch 2024-02-10T07:47:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:36 smithi155 ceph-mon[32688]: osdmap e429: 8 total, 8 up, 8 in 2024-02-10T07:47:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-75"}]: dispatch 2024-02-10T07:47:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-75"}]: dispatch 2024-02-10T07:47:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:37 smithi155 ceph-mon[32688]: pgmap v628: 356 pgs: 13 creating+activating, 343 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:47:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:37 smithi155 ceph-mon[32688]: osdmap e430: 8 total, 8 up, 8 in 2024-02-10T07:47:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:37 smithi070 ceph-mon[29548]: pgmap v628: 356 pgs: 13 creating+activating, 343 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:47:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:37 smithi070 ceph-mon[29548]: osdmap e430: 8 total, 8 up, 8 in 2024-02-10T07:47:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:37 smithi070 ceph-mon[39923]: pgmap v628: 356 pgs: 13 creating+activating, 343 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:47:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:37 smithi070 ceph-mon[39923]: osdmap e430: 8 total, 8 up, 8 in 2024-02-10T07:47:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[29548]: osdmap e431: 8 total, 8 up, 8 in 2024-02-10T07:47:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[29548]: pgmap v631: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[29548]: osdmap e432: 8 total, 8 up, 8 in 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[39923]: osdmap e431: 8 total, 8 up, 8 in 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[39923]: pgmap v631: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:38 smithi070 ceph-mon[39923]: osdmap e432: 8 total, 8 up, 8 in 2024-02-10T07:47:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:38 smithi155 ceph-mon[32688]: osdmap e431: 8 total, 8 up, 8 in 2024-02-10T07:47:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:38 smithi155 ceph-mon[32688]: pgmap v631: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:47:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:38 smithi155 ceph-mon[32688]: osdmap e432: 8 total, 8 up, 8 in 2024-02-10T07:47:39.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:47:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:47:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[29548]: osdmap e433: 8 total, 8 up, 8 in 2024-02-10T07:47:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:39 smithi070 ceph-mon[39923]: osdmap e433: 8 total, 8 up, 8 in 2024-02-10T07:47:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:39 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:39 smithi155 ceph-mon[32688]: osdmap e433: 8 total, 8 up, 8 in 2024-02-10T07:47:40.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:40.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-77"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-77"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[39923]: pgmap v634: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-77"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-77"}]: dispatch 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[29548]: pgmap v634: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:47:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:47:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-77"}]: dispatch 2024-02-10T07:47:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-77"}]: dispatch 2024-02-10T07:47:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:40 smithi155 ceph-mon[32688]: pgmap v634: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:47:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:41 smithi070 ceph-mon[29548]: osdmap e434: 8 total, 8 up, 8 in 2024-02-10T07:47:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:41 smithi070 ceph-mon[39923]: osdmap e434: 8 total, 8 up, 8 in 2024-02-10T07:47:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:41 smithi155 ceph-mon[32688]: osdmap e434: 8 total, 8 up, 8 in 2024-02-10T07:47:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:47:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:47:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[29548]: osdmap e435: 8 total, 8 up, 8 in 2024-02-10T07:47:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[29548]: pgmap v637: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[29548]: osdmap e436: 8 total, 8 up, 8 in 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[39923]: osdmap e435: 8 total, 8 up, 8 in 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[39923]: pgmap v637: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:42 smithi070 ceph-mon[39923]: osdmap e436: 8 total, 8 up, 8 in 2024-02-10T07:47:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:42 smithi155 ceph-mon[32688]: osdmap e435: 8 total, 8 up, 8 in 2024-02-10T07:47:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:47:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:42 smithi155 ceph-mon[32688]: pgmap v637: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:47:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:47:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:42 smithi155 ceph-mon[32688]: osdmap e436: 8 total, 8 up, 8 in 2024-02-10T07:47:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:43 smithi070 ceph-mon[29548]: osdmap e437: 8 total, 8 up, 8 in 2024-02-10T07:47:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:43 smithi070 ceph-mon[39923]: osdmap e437: 8 total, 8 up, 8 in 2024-02-10T07:47:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:47:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:43 smithi155 ceph-mon[32688]: osdmap e437: 8 total, 8 up, 8 in 2024-02-10T07:47:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:44 smithi070 ceph-mon[39923]: pgmap v640: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:47:44.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:44 smithi070 ceph-mon[29548]: pgmap v640: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:47:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:44 smithi155 ceph-mon[32688]: pgmap v640: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:47:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[39923]: osdmap e438: 8 total, 8 up, 8 in 2024-02-10T07:47:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.19"}]: dispatch 2024-02-10T07:47:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[39923]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.19"}]: dispatch 2024-02-10T07:47:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[29548]: osdmap e438: 8 total, 8 up, 8 in 2024-02-10T07:47:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.19"}]: dispatch 2024-02-10T07:47:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:45 smithi070 ceph-mon[29548]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.19"}]: dispatch 2024-02-10T07:47:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:45 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:47:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:45 smithi155 ceph-mon[32688]: osdmap e438: 8 total, 8 up, 8 in 2024-02-10T07:47:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.19"}]: dispatch 2024-02-10T07:47:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:45 smithi155 ceph-mon[32688]: from='mon.? v1:172.21.15.155:0/139194534' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.19"}]: dispatch 2024-02-10T07:47:46.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:46 smithi070 ceph-mon[39923]: 293.19 deep-scrub starts 2024-02-10T07:47:46.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:46 smithi070 ceph-mon[39923]: 293.19 deep-scrub ok 2024-02-10T07:47:46.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:46 smithi070 ceph-mon[39923]: pgmap v642: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:47:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:46 smithi070 ceph-mon[29548]: 293.19 deep-scrub starts 2024-02-10T07:47:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:46 smithi070 ceph-mon[29548]: 293.19 deep-scrub ok 2024-02-10T07:47:46.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:46 smithi070 ceph-mon[29548]: pgmap v642: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:47:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:46 smithi155 ceph-mon[32688]: 293.19 deep-scrub starts 2024-02-10T07:47:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:46 smithi155 ceph-mon[32688]: 293.19 deep-scrub ok 2024-02-10T07:47:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:46 smithi155 ceph-mon[32688]: pgmap v642: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:47:49.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:49 smithi070 ceph-mon[39923]: pgmap v643: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 852 B/s wr, 2 op/s 2024-02-10T07:47:49.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:49 smithi070 ceph-mon[29548]: pgmap v643: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 852 B/s wr, 2 op/s 2024-02-10T07:47:49.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:47:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:47:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:49 smithi155 ceph-mon[32688]: pgmap v643: 356 pgs: 356 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 852 B/s wr, 2 op/s 2024-02-10T07:47:50.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:50.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:50.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:47:51.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:51 smithi070 ceph-mon[39923]: pgmap v644: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 838 B/s wr, 3 op/s 2024-02-10T07:47:51.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:51 smithi070 ceph-mon[29548]: pgmap v644: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 838 B/s wr, 3 op/s 2024-02-10T07:47:51.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:51 smithi155 ceph-mon[32688]: pgmap v644: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 838 B/s wr, 3 op/s 2024-02-10T07:47:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:47:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:47:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:47:53.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:53 smithi070 ceph-mon[39923]: pgmap v645: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 739 B/s wr, 2 op/s 2024-02-10T07:47:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:53 smithi070 ceph-mon[29548]: pgmap v645: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 739 B/s wr, 2 op/s 2024-02-10T07:47:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:53 smithi155 ceph-mon[32688]: pgmap v645: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 739 B/s wr, 2 op/s 2024-02-10T07:47:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:55 smithi070 ceph-mon[29548]: pgmap v646: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T07:47:55.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:55.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:55 smithi070 ceph-mon[39923]: pgmap v646: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T07:47:55.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:55 smithi155 ceph-mon[32688]: pgmap v646: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T07:47:55.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:47:57.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:57 smithi070 ceph-mon[39923]: pgmap v647: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 906 B/s rd, 90 B/s wr, 1 op/s 2024-02-10T07:47:57.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:57 smithi070 ceph-mon[29548]: pgmap v647: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 906 B/s rd, 90 B/s wr, 1 op/s 2024-02-10T07:47:57.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:57 smithi155 ceph-mon[32688]: pgmap v647: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 906 B/s rd, 90 B/s wr, 1 op/s 2024-02-10T07:47:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:47:59 smithi070 ceph-mon[29548]: pgmap v648: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:47:59.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:47:59 smithi070 ceph-mon[39923]: pgmap v648: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:47:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:47:59 smithi155 ceph-mon[32688]: pgmap v648: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:47:59.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:47:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:48:00.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:00.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:01.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:01 smithi070 ceph-mon[29548]: pgmap v649: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:48:01.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:01 smithi070 ceph-mon[39923]: pgmap v649: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:48:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:01 smithi155 ceph-mon[32688]: pgmap v649: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T07:48:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:48:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:48:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:48:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:03 smithi070 ceph-mon[29548]: pgmap v650: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:48:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:03 smithi070 ceph-mon[39923]: pgmap v650: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:48:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:03 smithi155 ceph-mon[32688]: pgmap v650: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:48:04.991 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 07:48:04 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0[43929]: 2024-02-10T07:48:04.686+0000 7f5996fef700 -1 snap_mapper.add_oid found existing snaps mapped on 293:9827a41d:test-rados-api-smithi070-77569-80::foo:2, removing 2024-02-10T07:48:04.991 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 07:48:04 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1[49779]: 2024-02-10T07:48:04.686+0000 7fa880304700 -1 snap_mapper.add_oid found existing snaps mapped on 293:9827a41d:test-rados-api-smithi070-77569-80::foo:2, removing 2024-02-10T07:48:05.005 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 07:48:04 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T07:48:04.686+0000 7fb11ad9a700 -1 snap_mapper.add_oid found existing snaps mapped on 293:9827a41d:test-rados-api-smithi070-77569-80::foo:2, removing 2024-02-10T07:48:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[39923]: pgmap v651: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-79"}]: dispatch 2024-02-10T07:48:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-79"}]: dispatch 2024-02-10T07:48:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[29548]: pgmap v651: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-79"}]: dispatch 2024-02-10T07:48:05.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:05 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-79"}]: dispatch 2024-02-10T07:48:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:05 smithi155 ceph-mon[32688]: pgmap v651: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:05 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-79"}]: dispatch 2024-02-10T07:48:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:05 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-79"}]: dispatch 2024-02-10T07:48:06.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:06 smithi070 ceph-mon[29548]: osdmap e439: 8 total, 8 up, 8 in 2024-02-10T07:48:06.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:06 smithi070 ceph-mon[39923]: osdmap e439: 8 total, 8 up, 8 in 2024-02-10T07:48:06.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:06 smithi155 ceph-mon[32688]: osdmap e439: 8 total, 8 up, 8 in 2024-02-10T07:48:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[29548]: pgmap v653: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:48:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[29548]: osdmap e440: 8 total, 8 up, 8 in 2024-02-10T07:48:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:07.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[39923]: pgmap v653: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:48:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[39923]: osdmap e440: 8 total, 8 up, 8 in 2024-02-10T07:48:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:07.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:07 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:07 smithi155 ceph-mon[32688]: pgmap v653: 324 pgs: 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:48:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:07 smithi155 ceph-mon[32688]: osdmap e440: 8 total, 8 up, 8 in 2024-02-10T07:48:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:07 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:08.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[29548]: osdmap e441: 8 total, 8 up, 8 in 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[39923]: osdmap e441: 8 total, 8 up, 8 in 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:08.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:08 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:08 smithi155 ceph-mon[32688]: osdmap e441: 8 total, 8 up, 8 in 2024-02-10T07:48:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:08.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:08 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[29548]: pgmap v656: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:48:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[29548]: osdmap e442: 8 total, 8 up, 8 in 2024-02-10T07:48:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[39923]: pgmap v656: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:48:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[39923]: osdmap e442: 8 total, 8 up, 8 in 2024-02-10T07:48:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:09 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:09.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:48:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:48:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:09 smithi155 ceph-mon[32688]: pgmap v656: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:48:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:09 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:09 smithi155 ceph-mon[32688]: osdmap e442: 8 total, 8 up, 8 in 2024-02-10T07:48:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:09 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629eb30 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629f210 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629ed70 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629eea0 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629eff0 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629f340 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629f7d0 2024-02-10T07:48:10.425 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629f900 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629f100 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629fa30 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629fb80 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629f470 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629f5c0 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0150 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629fcb0 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629fde0 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629ff10 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0040 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0720 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0280 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a03d0 2024-02-10T07:48:10.426 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0500 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0850 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0d30 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0eb0 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629d6b0 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b2629d7e0 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1480 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0fe0 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1110 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1240 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1370 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1a50 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a15b0 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a16e0 2024-02-10T07:48:10.427 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1830 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1ba0 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2080 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2200 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0980 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a0ab0 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a27d0 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2330 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2460 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2590 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a26c0 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2da0 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2900 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2a30 2024-02-10T07:48:10.428 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2b80 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a2ed0 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3360 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3490 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1cd0 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1e20 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3a40 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a1f50 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3b70 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a35a0 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a36d0 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3800 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3930 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4140 2024-02-10T07:48:10.429 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3ca0 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3df0 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3f20 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4270 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4700 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4830 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3000 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a3130 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4e00 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4960 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4a90 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4bc0 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4cf0 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a53d0 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a4f30 2024-02-10T07:48:10.430 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5060 2024-02-10T07:48:10.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5190 2024-02-10T07:48:10.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a52c0 2024-02-10T07:48:10.431 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a59a0 2024-02-10T07:48:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[29548]: osdmap e443: 8 total, 8 up, 8 in 2024-02-10T07:48:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[39923]: osdmap e443: 8 total, 8 up, 8 in 2024-02-10T07:48:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:10 smithi155 ceph-mon[32688]: osdmap e443: 8 total, 8 up, 8 in 2024-02-10T07:48:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:10 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[29548]: pgmap v659: 356 pgs: 356 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-10T07:48:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[29548]: osdmap e444: 8 total, 8 up, 8 in 2024-02-10T07:48:11.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:11.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[39923]: pgmap v659: 356 pgs: 356 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-10T07:48:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[39923]: osdmap e444: 8 total, 8 up, 8 in 2024-02-10T07:48:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:11 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:11 smithi155 ceph-mon[32688]: pgmap v659: 356 pgs: 356 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-10T07:48:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:11 smithi155 ceph-mon[32688]: osdmap e444: 8 total, 8 up, 8 in 2024-02-10T07:48:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:11 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:12.484 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:48:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:48:12.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:12.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[29548]: osdmap e445: 8 total, 8 up, 8 in 2024-02-10T07:48:12.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T07:48:12.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T07:48:12.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:12.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[39923]: osdmap e445: 8 total, 8 up, 8 in 2024-02-10T07:48:12.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T07:48:12.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:12 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T07:48:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:12 smithi155 ceph-mon[32688]: osdmap e445: 8 total, 8 up, 8 in 2024-02-10T07:48:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T07:48:12.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:12 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T07:48:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:13 smithi155 ceph-mon[32688]: pgmap v662: 356 pgs: 356 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-10T07:48:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T07:48:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:13 smithi155 ceph-mon[32688]: osdmap e446: 8 total, 8 up, 8 in 2024-02-10T07:48:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T07:48:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:13 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T07:48:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[39923]: pgmap v662: 356 pgs: 356 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-10T07:48:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T07:48:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[39923]: osdmap e446: 8 total, 8 up, 8 in 2024-02-10T07:48:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T07:48:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T07:48:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[29548]: pgmap v662: 356 pgs: 356 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-10T07:48:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T07:48:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[29548]: osdmap e446: 8 total, 8 up, 8 in 2024-02-10T07:48:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T07:48:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:13 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T07:48:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:14 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T07:48:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:14 smithi155 ceph-mon[32688]: osdmap e447: 8 total, 8 up, 8 in 2024-02-10T07:48:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:14.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:14 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:14.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T07:48:14.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[39923]: osdmap e447: 8 total, 8 up, 8 in 2024-02-10T07:48:14.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:14.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:14.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T07:48:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[29548]: osdmap e447: 8 total, 8 up, 8 in 2024-02-10T07:48:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:14 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:15 smithi155 ceph-mon[32688]: pgmap v665: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-02-10T07:48:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:15 smithi155 ceph-mon[32688]: osdmap e448: 8 total, 8 up, 8 in 2024-02-10T07:48:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-81"}]: dispatch 2024-02-10T07:48:15.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:15 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-81"}]: dispatch 2024-02-10T07:48:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[39923]: pgmap v665: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-02-10T07:48:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[39923]: osdmap e448: 8 total, 8 up, 8 in 2024-02-10T07:48:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-81"}]: dispatch 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-81"}]: dispatch 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[29548]: pgmap v665: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[29548]: osdmap e448: 8 total, 8 up, 8 in 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-81"}]: dispatch 2024-02-10T07:48:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:15 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-81"}]: dispatch 2024-02-10T07:48:16.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:16 smithi070 ceph-mon[29548]: osdmap e449: 8 total, 8 up, 8 in 2024-02-10T07:48:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:16 smithi070 ceph-mon[39923]: osdmap e449: 8 total, 8 up, 8 in 2024-02-10T07:48:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:16 smithi155 ceph-mon[32688]: osdmap e449: 8 total, 8 up, 8 in 2024-02-10T07:48:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[29548]: pgmap v668: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-02-10T07:48:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[29548]: osdmap e450: 8 total, 8 up, 8 in 2024-02-10T07:48:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[39923]: pgmap v668: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-02-10T07:48:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[39923]: osdmap e450: 8 total, 8 up, 8 in 2024-02-10T07:48:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:17 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:17 smithi155 ceph-mon[32688]: pgmap v668: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-02-10T07:48:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:17 smithi155 ceph-mon[32688]: osdmap e450: 8 total, 8 up, 8 in 2024-02-10T07:48:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:17 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[29548]: osdmap e451: 8 total, 8 up, 8 in 2024-02-10T07:48:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:18.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[39923]: osdmap e451: 8 total, 8 up, 8 in 2024-02-10T07:48:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:18 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:18 smithi155 ceph-mon[32688]: osdmap e451: 8 total, 8 up, 8 in 2024-02-10T07:48:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:18 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:19.540 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:48:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:48:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[29548]: pgmap v671: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[29548]: osdmap e452: 8 total, 8 up, 8 in 2024-02-10T07:48:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:19.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[39923]: pgmap v671: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[39923]: osdmap e452: 8 total, 8 up, 8 in 2024-02-10T07:48:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:19 smithi155 ceph-mon[32688]: pgmap v671: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:19 smithi155 ceph-mon[32688]: osdmap e452: 8 total, 8 up, 8 in 2024-02-10T07:48:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[29548]: osdmap e453: 8 total, 8 up, 8 in 2024-02-10T07:48:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[39923]: osdmap e453: 8 total, 8 up, 8 in 2024-02-10T07:48:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:20.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:20 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:20 smithi155 ceph-mon[32688]: osdmap e453: 8 total, 8 up, 8 in 2024-02-10T07:48:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:20 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[29548]: pgmap v674: 356 pgs: 356 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-02-10T07:48:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[29548]: osdmap e454: 8 total, 8 up, 8 in 2024-02-10T07:48:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[39923]: pgmap v674: 356 pgs: 356 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-02-10T07:48:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[39923]: osdmap e454: 8 total, 8 up, 8 in 2024-02-10T07:48:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:21 smithi155 ceph-mon[32688]: pgmap v674: 356 pgs: 356 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-02-10T07:48:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:21 smithi155 ceph-mon[32688]: osdmap e454: 8 total, 8 up, 8 in 2024-02-10T07:48:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:48:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:48:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:48:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:22 smithi070 ceph-mon[29548]: osdmap e455: 8 total, 8 up, 8 in 2024-02-10T07:48:22.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:22.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:22 smithi070 ceph-mon[39923]: osdmap e455: 8 total, 8 up, 8 in 2024-02-10T07:48:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:22 smithi155 ceph-mon[32688]: osdmap e455: 8 total, 8 up, 8 in 2024-02-10T07:48:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:23 smithi070 ceph-mon[29548]: pgmap v677: 356 pgs: 356 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-02-10T07:48:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:23 smithi070 ceph-mon[29548]: osdmap e456: 8 total, 8 up, 8 in 2024-02-10T07:48:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:23 smithi070 ceph-mon[39923]: pgmap v677: 356 pgs: 356 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-02-10T07:48:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:23 smithi070 ceph-mon[39923]: osdmap e456: 8 total, 8 up, 8 in 2024-02-10T07:48:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:23 smithi155 ceph-mon[32688]: pgmap v677: 356 pgs: 356 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-02-10T07:48:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:23 smithi155 ceph-mon[32688]: osdmap e456: 8 total, 8 up, 8 in 2024-02-10T07:48:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[29548]: osdmap e457: 8 total, 8 up, 8 in 2024-02-10T07:48:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-83"}]: dispatch 2024-02-10T07:48:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-83"}]: dispatch 2024-02-10T07:48:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[39923]: osdmap e457: 8 total, 8 up, 8 in 2024-02-10T07:48:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-83"}]: dispatch 2024-02-10T07:48:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:24 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-83"}]: dispatch 2024-02-10T07:48:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:24 smithi155 ceph-mon[32688]: osdmap e457: 8 total, 8 up, 8 in 2024-02-10T07:48:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:24 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-83"}]: dispatch 2024-02-10T07:48:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:24 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-83"}]: dispatch 2024-02-10T07:48:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:25 smithi070 ceph-mon[29548]: pgmap v680: 356 pgs: 356 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-02-10T07:48:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:25 smithi070 ceph-mon[29548]: osdmap e458: 8 total, 8 up, 8 in 2024-02-10T07:48:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:25 smithi070 ceph-mon[39923]: pgmap v680: 356 pgs: 356 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-02-10T07:48:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:25 smithi070 ceph-mon[39923]: osdmap e458: 8 total, 8 up, 8 in 2024-02-10T07:48:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:25 smithi155 ceph-mon[32688]: pgmap v680: 356 pgs: 356 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-02-10T07:48:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:25 smithi155 ceph-mon[32688]: osdmap e458: 8 total, 8 up, 8 in 2024-02-10T07:48:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[39923]: osdmap e459: 8 total, 8 up, 8 in 2024-02-10T07:48:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:48:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[29548]: osdmap e459: 8 total, 8 up, 8 in 2024-02-10T07:48:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:48:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:26 smithi155 ceph-mon[32688]: osdmap e459: 8 total, 8 up, 8 in 2024-02-10T07:48:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:26 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:48:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: pgmap v683: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-10T07:48:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: osdmap e460: 8 total, 8 up, 8 in 2024-02-10T07:48:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: pgmap v683: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: osdmap e460: 8 total, 8 up, 8 in 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:27.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:27.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:48:27.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:48:27.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: pgmap v683: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: osdmap e460: 8 total, 8 up, 8 in 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:48:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:48:28.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:48:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[39923]: osdmap e461: 8 total, 8 up, 8 in 2024-02-10T07:48:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[39923]: pgmap v686: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[29548]: osdmap e461: 8 total, 8 up, 8 in 2024-02-10T07:48:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:28 smithi070 ceph-mon[29548]: pgmap v686: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:28 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:28 smithi155 ceph-mon[32688]: osdmap e461: 8 total, 8 up, 8 in 2024-02-10T07:48:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:28 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:28 smithi155 ceph-mon[32688]: pgmap v686: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:29.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:48:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:48:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[39923]: osdmap e462: 8 total, 8 up, 8 in 2024-02-10T07:48:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:29.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:29.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[29548]: osdmap e462: 8 total, 8 up, 8 in 2024-02-10T07:48:29.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:29.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:29.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:29 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_tier","val": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:48:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:29 smithi155 ceph-mon[32688]: osdmap e462: 8 total, 8 up, 8 in 2024-02-10T07:48:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:29 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:29 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[29548]: osdmap e463: 8 total, 8 up, 8 in 2024-02-10T07:48:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[29548]: pgmap v689: 356 pgs: 356 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[39923]: osdmap e463: 8 total, 8 up, 8 in 2024-02-10T07:48:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:30 smithi070 ceph-mon[39923]: pgmap v689: 356 pgs: 356 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:30 smithi155 ceph-mon[32688]: osdmap e463: 8 total, 8 up, 8 in 2024-02-10T07:48:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:30 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:30 smithi155 ceph-mon[32688]: pgmap v689: 356 pgs: 356 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:31 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:31 smithi070 ceph-mon[29548]: osdmap e464: 8 total, 8 up, 8 in 2024-02-10T07:48:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:31 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:31 smithi070 ceph-mon[39923]: osdmap e464: 8 total, 8 up, 8 in 2024-02-10T07:48:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:31 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:31 smithi155 ceph-mon[32688]: osdmap e464: 8 total, 8 up, 8 in 2024-02-10T07:48:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:48:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:48:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:48:32.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:32 smithi070 ceph-mon[29548]: osdmap e465: 8 total, 8 up, 8 in 2024-02-10T07:48:32.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:32 smithi070 ceph-mon[29548]: pgmap v692: 356 pgs: 356 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:32.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:32 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:32 smithi070 ceph-mon[39923]: osdmap e465: 8 total, 8 up, 8 in 2024-02-10T07:48:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:32 smithi070 ceph-mon[39923]: pgmap v692: 356 pgs: 356 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:32 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:32 smithi155 ceph-mon[32688]: osdmap e465: 8 total, 8 up, 8 in 2024-02-10T07:48:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:32 smithi155 ceph-mon[32688]: pgmap v692: 356 pgs: 356 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:32 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[39923]: osdmap e466: 8 total, 8 up, 8 in 2024-02-10T07:48:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-85"}]: dispatch 2024-02-10T07:48:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-85"}]: dispatch 2024-02-10T07:48:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[29548]: osdmap e466: 8 total, 8 up, 8 in 2024-02-10T07:48:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-85"}]: dispatch 2024-02-10T07:48:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-85"}]: dispatch 2024-02-10T07:48:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:33 smithi155 ceph-mon[32688]: osdmap e466: 8 total, 8 up, 8 in 2024-02-10T07:48:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-85"}]: dispatch 2024-02-10T07:48:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-85"}]: dispatch 2024-02-10T07:48:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:34 smithi070 ceph-mon[29548]: osdmap e467: 8 total, 8 up, 8 in 2024-02-10T07:48:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:34 smithi070 ceph-mon[29548]: pgmap v695: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:48:34.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:34 smithi070 ceph-mon[39923]: osdmap e467: 8 total, 8 up, 8 in 2024-02-10T07:48:34.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:34 smithi070 ceph-mon[39923]: pgmap v695: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:48:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:34 smithi155 ceph-mon[32688]: osdmap e467: 8 total, 8 up, 8 in 2024-02-10T07:48:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:34 smithi155 ceph-mon[32688]: pgmap v695: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18049 ms) 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22767 ms) 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3024 ms) 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3016 ms) 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3024 ms) 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.657 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3020 ms) 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4028 ms) 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39881 ms) 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17005 ms) 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4027 ms) 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.658 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3013 ms) 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25075 ms) 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15051 ms) 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5033 ms) 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4032 ms) 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: waiting for scrubs... 2024-02-10T07:48:35.659 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: done waiting 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24814 ms) 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10082 ms) 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9068 ms) 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9050 ms) 2024-02-10T07:48:35.660 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-10T07:48:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:35 smithi070 ceph-mon[39923]: osdmap e468: 8 total, 8 up, 8 in 2024-02-10T07:48:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:35 smithi070 ceph-mon[29548]: osdmap e468: 8 total, 8 up, 8 in 2024-02-10T07:48:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:35 smithi155 ceph-mon[32688]: osdmap e468: 8 total, 8 up, 8 in 2024-02-10T07:48:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[39923]: osdmap e469: 8 total, 8 up, 8 in 2024-02-10T07:48:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[39923]: pgmap v698: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:48:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[29548]: osdmap e469: 8 total, 8 up, 8 in 2024-02-10T07:48:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:36 smithi070 ceph-mon[29548]: pgmap v698: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:48:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:36 smithi155 ceph-mon[32688]: osdmap e469: 8 total, 8 up, 8 in 2024-02-10T07:48:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:36 smithi155 ceph-mon[32688]: pgmap v698: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T07:48:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:37 smithi155 ceph-mon[32688]: osdmap e470: 8 total, 8 up, 8 in 2024-02-10T07:48:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:37 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:37 smithi070 ceph-mon[39923]: osdmap e470: 8 total, 8 up, 8 in 2024-02-10T07:48:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:37 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:37 smithi070 ceph-mon[29548]: osdmap e470: 8 total, 8 up, 8 in 2024-02-10T07:48:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:37 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[29548]: osdmap e471: 8 total, 8 up, 8 in 2024-02-10T07:48:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]: dispatch 2024-02-10T07:48:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]: dispatch 2024-02-10T07:48:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[29548]: pgmap v701: 388 pgs: 64 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]': finished 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[29548]: osdmap e472: 8 total, 8 up, 8 in 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[39923]: osdmap e471: 8 total, 8 up, 8 in 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]: dispatch 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]: dispatch 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[39923]: pgmap v701: 388 pgs: 64 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]': finished 2024-02-10T07:48:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:38 smithi070 ceph-mon[39923]: osdmap e472: 8 total, 8 up, 8 in 2024-02-10T07:48:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:38 smithi155 ceph-mon[32688]: osdmap e471: 8 total, 8 up, 8 in 2024-02-10T07:48:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]: dispatch 2024-02-10T07:48:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]: dispatch 2024-02-10T07:48:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:38 smithi155 ceph-mon[32688]: pgmap v701: 388 pgs: 64 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_tier","val": "test-rados-api-smithi070-77569-89-test-flush"}]': finished 2024-02-10T07:48:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:38 smithi155 ceph-mon[32688]: osdmap e472: 8 total, 8 up, 8 in 2024-02-10T07:48:39.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:48:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: osdmap e473: 8 total, 8 up, 8 in 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: osdmap e473: 8 total, 8 up, 8 in 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: osdmap e473: 8 total, 8 up, 8 in 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:48:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[29548]: pgmap v704: 388 pgs: 388 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[29548]: osdmap e474: 8 total, 8 up, 8 in 2024-02-10T07:48:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[39923]: pgmap v704: 388 pgs: 388 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:40 smithi070 ceph-mon[39923]: osdmap e474: 8 total, 8 up, 8 in 2024-02-10T07:48:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:40 smithi155 ceph-mon[32688]: pgmap v704: 388 pgs: 388 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:48:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:40 smithi155 ceph-mon[32688]: osdmap e474: 8 total, 8 up, 8 in 2024-02-10T07:48:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:48:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:48:42.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[39923]: osdmap e475: 8 total, 8 up, 8 in 2024-02-10T07:48:42.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:42.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:42.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-87"}]: dispatch 2024-02-10T07:48:42.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-87"}]: dispatch 2024-02-10T07:48:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[39923]: pgmap v707: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[29548]: osdmap e475: 8 total, 8 up, 8 in 2024-02-10T07:48:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-87"}]: dispatch 2024-02-10T07:48:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-87"}]: dispatch 2024-02-10T07:48:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:42 smithi070 ceph-mon[29548]: pgmap v707: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:42 smithi155 ceph-mon[32688]: osdmap e475: 8 total, 8 up, 8 in 2024-02-10T07:48:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-87"}]: dispatch 2024-02-10T07:48:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-87"}]: dispatch 2024-02-10T07:48:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:42 smithi155 ceph-mon[32688]: pgmap v707: 356 pgs: 356 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:48:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:43 smithi070 ceph-mon[29548]: osdmap e476: 8 total, 8 up, 8 in 2024-02-10T07:48:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:43 smithi070 ceph-mon[39923]: osdmap e476: 8 total, 8 up, 8 in 2024-02-10T07:48:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:43 smithi155 ceph-mon[32688]: osdmap e476: 8 total, 8 up, 8 in 2024-02-10T07:48:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[39923]: osdmap e477: 8 total, 8 up, 8 in 2024-02-10T07:48:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[39923]: pgmap v710: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:48:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[29548]: osdmap e477: 8 total, 8 up, 8 in 2024-02-10T07:48:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:44 smithi070 ceph-mon[29548]: pgmap v710: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:48:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:44 smithi155 ceph-mon[32688]: osdmap e477: 8 total, 8 up, 8 in 2024-02-10T07:48:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:44 smithi155 ceph-mon[32688]: pgmap v710: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:48:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[29548]: osdmap e478: 8 total, 8 up, 8 in 2024-02-10T07:48:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[39923]: osdmap e478: 8 total, 8 up, 8 in 2024-02-10T07:48:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:45 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:45 smithi155 ceph-mon[32688]: osdmap e478: 8 total, 8 up, 8 in 2024-02-10T07:48:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:48:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:46 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:46 smithi070 ceph-mon[29548]: osdmap e479: 8 total, 8 up, 8 in 2024-02-10T07:48:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:46 smithi070 ceph-mon[29548]: pgmap v713: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:48:46.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:46 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:46 smithi070 ceph-mon[39923]: osdmap e479: 8 total, 8 up, 8 in 2024-02-10T07:48:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:46 smithi070 ceph-mon[39923]: pgmap v713: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:48:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:46 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:48:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:46 smithi155 ceph-mon[32688]: osdmap e479: 8 total, 8 up, 8 in 2024-02-10T07:48:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:46 smithi155 ceph-mon[32688]: pgmap v713: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:48:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:47 smithi070 ceph-mon[29548]: osdmap e480: 8 total, 8 up, 8 in 2024-02-10T07:48:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:47 smithi070 ceph-mon[39923]: osdmap e480: 8 total, 8 up, 8 in 2024-02-10T07:48:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:47 smithi155 ceph-mon[32688]: osdmap e480: 8 total, 8 up, 8 in 2024-02-10T07:48:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[29548]: osdmap e481: 8 total, 8 up, 8 in 2024-02-10T07:48:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-90"}]: dispatch 2024-02-10T07:48:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-90"}]: dispatch 2024-02-10T07:48:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[29548]: pgmap v716: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[39923]: osdmap e481: 8 total, 8 up, 8 in 2024-02-10T07:48:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-90"}]: dispatch 2024-02-10T07:48:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-90"}]: dispatch 2024-02-10T07:48:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:48 smithi070 ceph-mon[39923]: pgmap v716: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:48 smithi155 ceph-mon[32688]: osdmap e481: 8 total, 8 up, 8 in 2024-02-10T07:48:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-90"}]: dispatch 2024-02-10T07:48:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-90"}]: dispatch 2024-02-10T07:48:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:48 smithi155 ceph-mon[32688]: pgmap v716: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:49.713 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:48:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:48:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:49 smithi070 ceph-mon[29548]: osdmap e482: 8 total, 8 up, 8 in 2024-02-10T07:48:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:49 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:49 smithi070 ceph-mon[39923]: osdmap e482: 8 total, 8 up, 8 in 2024-02-10T07:48:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:49 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:49 smithi155 ceph-mon[32688]: osdmap e482: 8 total, 8 up, 8 in 2024-02-10T07:48:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:49 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:48:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[29548]: osdmap e483: 8 total, 8 up, 8 in 2024-02-10T07:48:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[29548]: pgmap v719: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:48:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[39923]: osdmap e483: 8 total, 8 up, 8 in 2024-02-10T07:48:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:50 smithi070 ceph-mon[39923]: pgmap v719: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:48:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:50 smithi155 ceph-mon[32688]: osdmap e483: 8 total, 8 up, 8 in 2024-02-10T07:48:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:50 smithi155 ceph-mon[32688]: pgmap v719: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:48:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[29548]: osdmap e484: 8 total, 8 up, 8 in 2024-02-10T07:48:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[39923]: osdmap e484: 8 total, 8 up, 8 in 2024-02-10T07:48:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:51 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:51 smithi155 ceph-mon[32688]: osdmap e484: 8 total, 8 up, 8 in 2024-02-10T07:48:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:48:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:48:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:48:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:52 smithi070 ceph-mon[29548]: osdmap e485: 8 total, 8 up, 8 in 2024-02-10T07:48:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:52 smithi070 ceph-mon[29548]: pgmap v722: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:48:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:52 smithi070 ceph-mon[39923]: osdmap e485: 8 total, 8 up, 8 in 2024-02-10T07:48:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:52 smithi070 ceph-mon[39923]: pgmap v722: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:48:53.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:52 smithi155 ceph-mon[32688]: osdmap e485: 8 total, 8 up, 8 in 2024-02-10T07:48:53.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:52 smithi155 ceph-mon[32688]: pgmap v722: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T07:48:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:53 smithi155 ceph-mon[32688]: osdmap e486: 8 total, 8 up, 8 in 2024-02-10T07:48:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-92"}]: dispatch 2024-02-10T07:48:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-92"}]: dispatch 2024-02-10T07:48:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[39923]: osdmap e486: 8 total, 8 up, 8 in 2024-02-10T07:48:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-92"}]: dispatch 2024-02-10T07:48:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-92"}]: dispatch 2024-02-10T07:48:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[29548]: osdmap e486: 8 total, 8 up, 8 in 2024-02-10T07:48:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:48:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-92"}]: dispatch 2024-02-10T07:48:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-92"}]: dispatch 2024-02-10T07:48:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:54 smithi155 ceph-mon[32688]: osdmap e487: 8 total, 8 up, 8 in 2024-02-10T07:48:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:54 smithi155 ceph-mon[32688]: pgmap v725: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:48:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:54 smithi070 ceph-mon[29548]: osdmap e487: 8 total, 8 up, 8 in 2024-02-10T07:48:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:54 smithi070 ceph-mon[29548]: pgmap v725: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:48:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:54 smithi070 ceph-mon[39923]: osdmap e487: 8 total, 8 up, 8 in 2024-02-10T07:48:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:54 smithi070 ceph-mon[39923]: pgmap v725: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:48:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:55 smithi155 ceph-mon[32688]: osdmap e488: 8 total, 8 up, 8 in 2024-02-10T07:48:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[39923]: osdmap e488: 8 total, 8 up, 8 in 2024-02-10T07:48:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[29548]: osdmap e488: 8 total, 8 up, 8 in 2024-02-10T07:48:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:48:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:48:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[29548]: osdmap e489: 8 total, 8 up, 8 in 2024-02-10T07:48:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[29548]: pgmap v728: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:48:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[39923]: osdmap e489: 8 total, 8 up, 8 in 2024-02-10T07:48:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:56 smithi070 ceph-mon[39923]: pgmap v728: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:48:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:48:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:56 smithi155 ceph-mon[32688]: osdmap e489: 8 total, 8 up, 8 in 2024-02-10T07:48:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:48:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:56 smithi155 ceph-mon[32688]: pgmap v728: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T07:48:58.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:57 smithi070 ceph-mon[39923]: osdmap e490: 8 total, 8 up, 8 in 2024-02-10T07:48:58.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:57 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:57 smithi070 ceph-mon[29548]: osdmap e490: 8 total, 8 up, 8 in 2024-02-10T07:48:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:57 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:57 smithi155 ceph-mon[32688]: osdmap e490: 8 total, 8 up, 8 in 2024-02-10T07:48:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:57 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:48:59.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:58 smithi070 ceph-mon[39923]: osdmap e491: 8 total, 8 up, 8 in 2024-02-10T07:48:59.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:58 smithi070 ceph-mon[39923]: pgmap v731: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:58 smithi070 ceph-mon[29548]: osdmap e491: 8 total, 8 up, 8 in 2024-02-10T07:48:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:58 smithi070 ceph-mon[29548]: pgmap v731: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:58 smithi155 ceph-mon[32688]: osdmap e491: 8 total, 8 up, 8 in 2024-02-10T07:48:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:58 smithi155 ceph-mon[32688]: pgmap v731: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:48:59.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:48:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:49:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:48:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:00.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:48:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:48:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:00 smithi070 ceph-mon[29548]: pgmap v732: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:49:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:00 smithi070 ceph-mon[39923]: pgmap v732: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:49:01.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:00 smithi155 ceph-mon[32688]: pgmap v732: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:49:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:49:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:49:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:49:03.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:03 smithi070 ceph-mon[29548]: pgmap v733: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:49:03.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:03 smithi070 ceph-mon[39923]: pgmap v733: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:49:03.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:03 smithi155 ceph-mon[32688]: pgmap v733: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T07:49:05.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:05 smithi070 ceph-mon[29548]: pgmap v734: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:49:05.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:05 smithi070 ceph-mon[39923]: pgmap v734: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:49:05.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:05 smithi155 ceph-mon[32688]: pgmap v734: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T07:49:07.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:07 smithi070 ceph-mon[29548]: pgmap v735: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 890 B/s wr, 3 op/s 2024-02-10T07:49:07.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:07 smithi070 ceph-mon[39923]: pgmap v735: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 890 B/s wr, 3 op/s 2024-02-10T07:49:07.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:07 smithi155 ceph-mon[32688]: pgmap v735: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 890 B/s wr, 3 op/s 2024-02-10T07:49:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:09 smithi070 ceph-mon[29548]: pgmap v736: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 803 B/s wr, 3 op/s 2024-02-10T07:49:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:09 smithi070 ceph-mon[29548]: osdmap e492: 8 total, 8 up, 8 in 2024-02-10T07:49:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:09 smithi070 ceph-mon[39923]: pgmap v736: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 803 B/s wr, 3 op/s 2024-02-10T07:49:09.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:09 smithi070 ceph-mon[39923]: osdmap e492: 8 total, 8 up, 8 in 2024-02-10T07:49:09.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:49:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:49:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:09 smithi155 ceph-mon[32688]: pgmap v736: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 803 B/s wr, 3 op/s 2024-02-10T07:49:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:09 smithi155 ceph-mon[32688]: osdmap e492: 8 total, 8 up, 8 in 2024-02-10T07:49:10.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:10.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:11.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:11 smithi070 ceph-mon[29548]: pgmap v738: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:11.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:11 smithi070 ceph-mon[39923]: pgmap v738: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:11 smithi155 ceph-mon[32688]: pgmap v738: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:49:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:49:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:49:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:13 smithi070 ceph-mon[39923]: pgmap v739: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:13 smithi070 ceph-mon[39923]: osdmap e493: 8 total, 8 up, 8 in 2024-02-10T07:49:13.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:13 smithi070 ceph-mon[29548]: pgmap v739: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:13.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:13 smithi070 ceph-mon[29548]: osdmap e493: 8 total, 8 up, 8 in 2024-02-10T07:49:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:13 smithi155 ceph-mon[32688]: pgmap v739: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:13 smithi155 ceph-mon[32688]: osdmap e493: 8 total, 8 up, 8 in 2024-02-10T07:49:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:15 smithi070 ceph-mon[29548]: pgmap v741: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:15 smithi070 ceph-mon[39923]: pgmap v741: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:15 smithi155 ceph-mon[32688]: pgmap v741: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:17 smithi070 ceph-mon[29548]: pgmap v742: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:17.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:17 smithi070 ceph-mon[39923]: pgmap v742: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:17 smithi155 ceph-mon[32688]: pgmap v742: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[29548]: pgmap v743: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-94"}]: dispatch 2024-02-10T07:49:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-94"}]: dispatch 2024-02-10T07:49:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[39923]: pgmap v743: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-94"}]: dispatch 2024-02-10T07:49:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:19 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-94"}]: dispatch 2024-02-10T07:49:19.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:49:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:49:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:19 smithi155 ceph-mon[32688]: pgmap v743: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-94"}]: dispatch 2024-02-10T07:49:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:19 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-94"}]: dispatch 2024-02-10T07:49:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:20 smithi070 ceph-mon[29548]: osdmap e494: 8 total, 8 up, 8 in 2024-02-10T07:49:20.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:20.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:20 smithi070 ceph-mon[39923]: osdmap e494: 8 total, 8 up, 8 in 2024-02-10T07:49:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:20 smithi155 ceph-mon[32688]: osdmap e494: 8 total, 8 up, 8 in 2024-02-10T07:49:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[39923]: pgmap v745: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[39923]: osdmap e495: 8 total, 8 up, 8 in 2024-02-10T07:49:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:21.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[29548]: pgmap v745: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:21.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[29548]: osdmap e495: 8 total, 8 up, 8 in 2024-02-10T07:49:21.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:21.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:21 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:21 smithi155 ceph-mon[32688]: pgmap v745: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:21 smithi155 ceph-mon[32688]: osdmap e495: 8 total, 8 up, 8 in 2024-02-10T07:49:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:21 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:22.445 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:49:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:49:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:49:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:22 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:22 smithi070 ceph-mon[39923]: osdmap e496: 8 total, 8 up, 8 in 2024-02-10T07:49:22.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:49:22.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:22 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:22 smithi070 ceph-mon[29548]: osdmap e496: 8 total, 8 up, 8 in 2024-02-10T07:49:22.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:49:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:22 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:22 smithi155 ceph-mon[32688]: osdmap e496: 8 total, 8 up, 8 in 2024-02-10T07:49:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:49:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:23 smithi070 ceph-mon[29548]: pgmap v748: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:23.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:23 smithi070 ceph-mon[29548]: osdmap e497: 8 total, 8 up, 8 in 2024-02-10T07:49:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:23 smithi070 ceph-mon[39923]: pgmap v748: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:23.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:23 smithi070 ceph-mon[39923]: osdmap e497: 8 total, 8 up, 8 in 2024-02-10T07:49:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:23 smithi155 ceph-mon[32688]: pgmap v748: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:23 smithi155 ceph-mon[32688]: osdmap e497: 8 total, 8 up, 8 in 2024-02-10T07:49:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:25 smithi070 ceph-mon[29548]: pgmap v750: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-10T07:49:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:25 smithi070 ceph-mon[39923]: pgmap v750: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-10T07:49:25.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:25 smithi155 ceph-mon[32688]: pgmap v750: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-10T07:49:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:27.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: pgmap v751: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 852 B/s wr, 2 op/s 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: pgmap v751: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 852 B/s wr, 2 op/s 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:49:27.744 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: pgmap v751: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 852 B/s wr, 2 op/s 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:49:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:49:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:49:27.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:49:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:28 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi070 to 3248M 2024-02-10T07:49:28.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:28 smithi070 ceph-mon[29548]: Adjusting osd_memory_target on smithi155 to 3248M 2024-02-10T07:49:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:28 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi070 to 3248M 2024-02-10T07:49:28.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:28 smithi070 ceph-mon[39923]: Adjusting osd_memory_target on smithi155 to 3248M 2024-02-10T07:49:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:28 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi070 to 3248M 2024-02-10T07:49:28.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:28 smithi155 ceph-mon[32688]: Adjusting osd_memory_target on smithi155 to 3248M 2024-02-10T07:49:29.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:29 smithi070 ceph-mon[39923]: pgmap v752: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 678 B/s rd, 678 B/s wr, 2 op/s 2024-02-10T07:49:29.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:29 smithi070 ceph-mon[29548]: pgmap v752: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 678 B/s rd, 678 B/s wr, 2 op/s 2024-02-10T07:49:29.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:49:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:49:29.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:29 smithi155 ceph-mon[32688]: pgmap v752: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 678 B/s rd, 678 B/s wr, 2 op/s 2024-02-10T07:49:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:31 smithi070 ceph-mon[29548]: pgmap v753: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 599 B/s wr, 2 op/s 2024-02-10T07:49:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:31 smithi070 ceph-mon[39923]: pgmap v753: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 599 B/s wr, 2 op/s 2024-02-10T07:49:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:31 smithi155 ceph-mon[32688]: pgmap v753: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 599 B/s wr, 2 op/s 2024-02-10T07:49:32.474 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:49:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:49:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:49:32.741 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 07:49:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3[61781]: 2024-02-10T07:49:32.473+0000 7f2ad6f3d700 -1 snap_mapper.add_oid found existing snaps mapped on 101:5179f712:test-rados-api-smithi070-77569-97::foo:21, removing 2024-02-10T07:49:32.755 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 07:49:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T07:49:32.473+0000 7f4969d22700 -1 snap_mapper.add_oid found existing snaps mapped on 101:5179f712:test-rados-api-smithi070-77569-97::foo:21, removing 2024-02-10T07:49:32.755 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 07:49:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T07:49:32.473+0000 7fac3cabf700 -1 snap_mapper.add_oid found existing snaps mapped on 101:5179f712:test-rados-api-smithi070-77569-97::foo:21, removing 2024-02-10T07:49:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[29548]: pgmap v754: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:49:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-96"}]: dispatch 2024-02-10T07:49:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-96"}]: dispatch 2024-02-10T07:49:33.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[39923]: pgmap v754: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:49:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-96"}]: dispatch 2024-02-10T07:49:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:33 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-96"}]: dispatch 2024-02-10T07:49:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:33 smithi155 ceph-mon[32688]: pgmap v754: 356 pgs: 356 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:49:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-96"}]: dispatch 2024-02-10T07:49:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:33 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-96"}]: dispatch 2024-02-10T07:49:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:34 smithi070 ceph-mon[29548]: osdmap e498: 8 total, 8 up, 8 in 2024-02-10T07:49:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:34 smithi070 ceph-mon[39923]: osdmap e498: 8 total, 8 up, 8 in 2024-02-10T07:49:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:34 smithi155 ceph-mon[32688]: osdmap e498: 8 total, 8 up, 8 in 2024-02-10T07:49:35.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[39923]: pgmap v756: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-02-10T07:49:35.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[39923]: osdmap e499: 8 total, 8 up, 8 in 2024-02-10T07:49:35.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:35.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[29548]: pgmap v756: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-02-10T07:49:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[29548]: osdmap e499: 8 total, 8 up, 8 in 2024-02-10T07:49:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:35.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:35 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:35 smithi155 ceph-mon[32688]: pgmap v756: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-02-10T07:49:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:35 smithi155 ceph-mon[32688]: osdmap e499: 8 total, 8 up, 8 in 2024-02-10T07:49:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:35 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:36 smithi155 ceph-mon[32688]: osdmap e500: 8 total, 8 up, 8 in 2024-02-10T07:49:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:36 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[39923]: osdmap e500: 8 total, 8 up, 8 in 2024-02-10T07:49:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[29548]: osdmap e500: 8 total, 8 up, 8 in 2024-02-10T07:49:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:36 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:37 smithi155 ceph-mon[32688]: pgmap v759: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:49:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:37 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:49:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:37 smithi155 ceph-mon[32688]: osdmap e501: 8 total, 8 up, 8 in 2024-02-10T07:49:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]: dispatch 2024-02-10T07:49:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:37 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]: dispatch 2024-02-10T07:49:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[39923]: pgmap v759: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:49:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:49:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[39923]: osdmap e501: 8 total, 8 up, 8 in 2024-02-10T07:49:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]: dispatch 2024-02-10T07:49:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]: dispatch 2024-02-10T07:49:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[29548]: pgmap v759: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:49:37.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:49:37.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[29548]: osdmap e501: 8 total, 8 up, 8 in 2024-02-10T07:49:37.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]: dispatch 2024-02-10T07:49:37.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:37 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]: dispatch 2024-02-10T07:49:38.439 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5500 2024-02-10T07:49:38.439 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5630 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5780 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5ad0 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5f60 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6090 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5890 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a61c0 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6310 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5c00 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5d10 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6900 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6460 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6590 2024-02-10T07:49:38.440 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a5e20 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a66c0 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6ed0 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6a30 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6b80 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a6cd0 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7000 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7490 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a75c0 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7b90 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7130 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7260 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a76f0 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7820 2024-02-10T07:49:38.441 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7950 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7a80 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8180 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7ce0 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7e10 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a7f60 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a82b0 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8740 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8870 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a43a0 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a44d0 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8e40 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a89a0 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8ad0 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8c00 2024-02-10T07:49:38.442 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8d30 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a9410 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a8f70 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a90a0 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a91f0 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a9540 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a99d0 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 0x564b262a9b00 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.443 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.444 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.445 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.446 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.446 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.446 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.446 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.446 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:49:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:38 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:49:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]': finished 2024-02-10T07:49:38.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:38 smithi155 ceph-mon[32688]: osdmap e502: 8 total, 8 up, 8 in 2024-02-10T07:49:38.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:38.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:38.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:38 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]': finished 2024-02-10T07:49:38.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:38 smithi155 ceph-mon[32688]: osdmap e503: 8 total, 8 up, 8 in 2024-02-10T07:49:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:49:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]': finished 2024-02-10T07:49:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[39923]: osdmap e502: 8 total, 8 up, 8 in 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]': finished 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[39923]: osdmap e503: 8 total, 8 up, 8 in 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-98", "mode": "writeback"}]': finished 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[29548]: osdmap e502: 8 total, 8 up, 8 in 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-98"}]': finished 2024-02-10T07:49:38.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:38 smithi070 ceph-mon[29548]: osdmap e503: 8 total, 8 up, 8 in 2024-02-10T07:49:39.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:49:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:49:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:39 smithi155 ceph-mon[32688]: pgmap v762: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:49:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:49:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:39 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:49:39.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:39 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:49:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[29548]: pgmap v762: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:49:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:49:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:49:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:49:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[39923]: pgmap v762: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:49:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:49:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:49:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:39 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:49:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[29548]: osdmap e504: 8 total, 8 up, 8 in 2024-02-10T07:49:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]': finished 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[39923]: osdmap e504: 8 total, 8 up, 8 in 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:40 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]': finished 2024-02-10T07:49:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:40 smithi155 ceph-mon[32688]: osdmap e504: 8 total, 8 up, 8 in 2024-02-10T07:49:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T07:49:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T07:49:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:40 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_count","val": "1"}]': finished 2024-02-10T07:49:41.505 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:49:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=cleanup t=2024-02-10T07:49:41.015027868Z level=info msg="Completed cleanup jobs" duration=2.388163ms 2024-02-10T07:49:41.505 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:49:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=sqlstore.transactions t=2024-02-10T07:49:41.023358566Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-10T07:49:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[39923]: pgmap v765: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:49:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[39923]: osdmap e505: 8 total, 8 up, 8 in 2024-02-10T07:49:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:49:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[29548]: pgmap v765: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:49:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[29548]: osdmap e505: 8 total, 8 up, 8 in 2024-02-10T07:49:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:41 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:49:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:41 smithi155 ceph-mon[32688]: pgmap v765: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:49:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:41 smithi155 ceph-mon[32688]: osdmap e505: 8 total, 8 up, 8 in 2024-02-10T07:49:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:41 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:41 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:49:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:49:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:49:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[29548]: osdmap e506: 8 total, 8 up, 8 in 2024-02-10T07:49:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T07:49:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T07:49:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]': finished 2024-02-10T07:49:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[39923]: osdmap e506: 8 total, 8 up, 8 in 2024-02-10T07:49:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T07:49:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T07:49:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:42 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]': finished 2024-02-10T07:49:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:42 smithi155 ceph-mon[32688]: osdmap e506: 8 total, 8 up, 8 in 2024-02-10T07:49:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T07:49:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T07:49:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:42 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-98","var": "target_max_objects","val": "250"}]': finished 2024-02-10T07:49:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[39923]: pgmap v768: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:49:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[39923]: osdmap e507: 8 total, 8 up, 8 in 2024-02-10T07:49:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[29548]: pgmap v768: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:49:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[29548]: osdmap e507: 8 total, 8 up, 8 in 2024-02-10T07:49:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:43 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:43 smithi155 ceph-mon[32688]: pgmap v768: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:49:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:43 smithi155 ceph-mon[32688]: osdmap e507: 8 total, 8 up, 8 in 2024-02-10T07:49:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:43 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:49:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[39923]: osdmap e508: 8 total, 8 up, 8 in 2024-02-10T07:49:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:49:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[29548]: osdmap e508: 8 total, 8 up, 8 in 2024-02-10T07:49:44.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:44.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:44 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:49:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:44 smithi155 ceph-mon[32688]: osdmap e508: 8 total, 8 up, 8 in 2024-02-10T07:49:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:44 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]: dispatch 2024-02-10T07:49:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:45 smithi070 ceph-mon[39923]: pgmap v771: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:45 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]': finished 2024-02-10T07:49:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:45 smithi070 ceph-mon[39923]: osdmap e509: 8 total, 8 up, 8 in 2024-02-10T07:49:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:45 smithi070 ceph-mon[29548]: pgmap v771: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:45 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]': finished 2024-02-10T07:49:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:45 smithi070 ceph-mon[29548]: osdmap e509: 8 total, 8 up, 8 in 2024-02-10T07:49:46.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:45 smithi155 ceph-mon[32688]: pgmap v771: 356 pgs: 356 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:49:46.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:45 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-98"}]': finished 2024-02-10T07:49:46.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:45 smithi155 ceph-mon[32688]: osdmap e509: 8 total, 8 up, 8 in 2024-02-10T07:49:46.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:46 smithi070 ceph-mon[39923]: osdmap e510: 8 total, 8 up, 8 in 2024-02-10T07:49:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:46 smithi070 ceph-mon[29548]: osdmap e510: 8 total, 8 up, 8 in 2024-02-10T07:49:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:46 smithi155 ceph-mon[32688]: osdmap e510: 8 total, 8 up, 8 in 2024-02-10T07:49:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[29548]: pgmap v774: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[29548]: osdmap e511: 8 total, 8 up, 8 in 2024-02-10T07:49:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[39923]: pgmap v774: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[39923]: osdmap e511: 8 total, 8 up, 8 in 2024-02-10T07:49:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:47 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:47 smithi155 ceph-mon[32688]: pgmap v774: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:47 smithi155 ceph-mon[32688]: osdmap e511: 8 total, 8 up, 8 in 2024-02-10T07:49:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:47 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[29548]: osdmap e512: 8 total, 8 up, 8 in 2024-02-10T07:49:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:48.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[39923]: osdmap e512: 8 total, 8 up, 8 in 2024-02-10T07:49:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:48 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:49:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:48 smithi155 ceph-mon[32688]: osdmap e512: 8 total, 8 up, 8 in 2024-02-10T07:49:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:48 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:49:49.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:49:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:49:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:49 smithi155 ceph-mon[32688]: pgmap v777: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:49.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:49:49.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:49 smithi155 ceph-mon[32688]: osdmap e513: 8 total, 8 up, 8 in 2024-02-10T07:49:49.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:49.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:49 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[29548]: pgmap v777: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:49:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[29548]: osdmap e513: 8 total, 8 up, 8 in 2024-02-10T07:49:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:49.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[39923]: pgmap v777: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:49:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[39923]: osdmap e513: 8 total, 8 up, 8 in 2024-02-10T07:49:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:49 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]': finished 2024-02-10T07:49:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[39923]: osdmap e514: 8 total, 8 up, 8 in 2024-02-10T07:49:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]: dispatch 2024-02-10T07:49:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]: dispatch 2024-02-10T07:49:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]': finished 2024-02-10T07:49:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[29548]: osdmap e514: 8 total, 8 up, 8 in 2024-02-10T07:49:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]: dispatch 2024-02-10T07:49:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:50 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]: dispatch 2024-02-10T07:49:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-6", "overlaypool": "test-rados-api-smithi070-77569-100"}]': finished 2024-02-10T07:49:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:50 smithi155 ceph-mon[32688]: osdmap e514: 8 total, 8 up, 8 in 2024-02-10T07:49:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]: dispatch 2024-02-10T07:49:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:50 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]: dispatch 2024-02-10T07:49:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[29548]: pgmap v780: 356 pgs: 356 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:49:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]': finished 2024-02-10T07:49:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[29548]: osdmap e515: 8 total, 8 up, 8 in 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[39923]: pgmap v780: 356 pgs: 356 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]': finished 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[39923]: osdmap e515: 8 total, 8 up, 8 in 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:49:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:51 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:49:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:51 smithi155 ceph-mon[32688]: pgmap v780: 356 pgs: 356 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:51 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:49:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-100", "mode": "writeback"}]': finished 2024-02-10T07:49:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:51 smithi155 ceph-mon[32688]: osdmap e515: 8 total, 8 up, 8 in 2024-02-10T07:49:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:49:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:51 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:49:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:49:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:49:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:49:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[29548]: osdmap e516: 8 total, 8 up, 8 in 2024-02-10T07:49:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:49:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[39923]: osdmap e516: 8 total, 8 up, 8 in 2024-02-10T07:49:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:52 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:52 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:49:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:52 smithi155 ceph-mon[32688]: osdmap e516: 8 total, 8 up, 8 in 2024-02-10T07:49:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:52 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:49:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[29548]: pgmap v783: 356 pgs: 356 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:49:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[29548]: osdmap e517: 8 total, 8 up, 8 in 2024-02-10T07:49:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:49:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:49:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[39923]: pgmap v783: 356 pgs: 356 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:49:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[39923]: osdmap e517: 8 total, 8 up, 8 in 2024-02-10T07:49:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:49:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:53 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:49:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:53 smithi155 ceph-mon[32688]: pgmap v783: 356 pgs: 356 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:49:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:53 smithi155 ceph-mon[32688]: osdmap e517: 8 total, 8 up, 8 in 2024-02-10T07:49:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:49:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:53 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:49:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:49:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:49:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[29548]: osdmap e518: 8 total, 8 up, 8 in 2024-02-10T07:49:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T07:49:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T07:49:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:49:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:49:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[39923]: osdmap e518: 8 total, 8 up, 8 in 2024-02-10T07:49:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T07:49:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:54 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T07:49:55.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:54 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:49:55.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:49:55.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:54 smithi155 ceph-mon[32688]: osdmap e518: 8 total, 8 up, 8 in 2024-02-10T07:49:55.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T07:49:55.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:54 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T07:49:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[29548]: pgmap v786: 356 pgs: 356 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T07:49:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[29548]: osdmap e519: 8 total, 8 up, 8 in 2024-02-10T07:49:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[39923]: pgmap v786: 356 pgs: 356 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T07:49:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[39923]: osdmap e519: 8 total, 8 up, 8 in 2024-02-10T07:49:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:55 smithi155 ceph-mon[32688]: pgmap v786: 356 pgs: 356 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T07:49:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:55 smithi155 ceph-mon[32688]: osdmap e519: 8 total, 8 up, 8 in 2024-02-10T07:49:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:55 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:49:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:49:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:49:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[29548]: osdmap e520: 8 total, 8 up, 8 in 2024-02-10T07:49:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[29548]: pgmap v789: 356 pgs: 356 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]': finished 2024-02-10T07:49:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:49:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[39923]: osdmap e520: 8 total, 8 up, 8 in 2024-02-10T07:49:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[39923]: pgmap v789: 356 pgs: 356 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:56 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]': finished 2024-02-10T07:49:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]': finished 2024-02-10T07:49:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:56 smithi155 ceph-mon[32688]: osdmap e520: 8 total, 8 up, 8 in 2024-02-10T07:49:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]: dispatch 2024-02-10T07:49:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:56 smithi155 ceph-mon[32688]: pgmap v789: 356 pgs: 356 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:49:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:56 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-100"}]': finished 2024-02-10T07:49:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:57 smithi070 ceph-mon[29548]: osdmap e521: 8 total, 8 up, 8 in 2024-02-10T07:49:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:57 smithi070 ceph-mon[39923]: osdmap e521: 8 total, 8 up, 8 in 2024-02-10T07:49:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:57 smithi155 ceph-mon[32688]: osdmap e521: 8 total, 8 up, 8 in 2024-02-10T07:49:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:58 smithi070 ceph-mon[29548]: osdmap e522: 8 total, 8 up, 8 in 2024-02-10T07:49:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:58 smithi070 ceph-mon[29548]: pgmap v792: 324 pgs: 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:58.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:58 smithi070 ceph-mon[39923]: osdmap e522: 8 total, 8 up, 8 in 2024-02-10T07:49:58.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:58 smithi070 ceph-mon[39923]: pgmap v792: 324 pgs: 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:58 smithi155 ceph-mon[32688]: osdmap e522: 8 total, 8 up, 8 in 2024-02-10T07:49:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:58 smithi155 ceph-mon[32688]: pgmap v792: 324 pgs: 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:49:59.624 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:49:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:49:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[29548]: osdmap e523: 8 total, 8 up, 8 in 2024-02-10T07:49:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:49:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[39923]: osdmap e523: 8 total, 8 up, 8 in 2024-02-10T07:49:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:49:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:49:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:59 smithi155 ceph-mon[32688]: osdmap e523: 8 total, 8 up, 8 in 2024-02-10T07:50:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:59 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:49:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:00.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:00.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: osdmap e524: 8 total, 8 up, 8 in 2024-02-10T07:50:00.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: pgmap v795: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: application not enabled on pool 'WatchNotifysmithi070-78654-1' 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: application not enabled on pool 'SnapListsmithi070-78656-1' 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: application not enabled on pool 'poolrace.1804289383' 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: application not enabled on pool 'poolrace2.846930886' 2024-02-10T07:50:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[29548]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: osdmap e524: 8 total, 8 up, 8 in 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: pgmap v795: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: application not enabled on pool 'WatchNotifysmithi070-78654-1' 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: application not enabled on pool 'SnapListsmithi070-78656-1' 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: application not enabled on pool 'poolrace.1804289383' 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: application not enabled on pool 'poolrace2.846930886' 2024-02-10T07:50:00.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:00 smithi070 ceph-mon[39923]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: osdmap e524: 8 total, 8 up, 8 in 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: pgmap v795: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T07:50:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-10T07:50:01.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T07:50:01.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: application not enabled on pool 'WatchNotifysmithi070-78654-1' 2024-02-10T07:50:01.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: application not enabled on pool 'SnapListsmithi070-78656-1' 2024-02-10T07:50:01.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: application not enabled on pool 'poolrace.1804289383' 2024-02-10T07:50:01.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: application not enabled on pool 'poolrace2.846930886' 2024-02-10T07:50:01.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:00 smithi155 ceph-mon[32688]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T07:50:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:50:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[39923]: osdmap e525: 8 total, 8 up, 8 in 2024-02-10T07:50:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:50:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:50:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:50:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[29548]: osdmap e525: 8 total, 8 up, 8 in 2024-02-10T07:50:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:50:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:01 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:50:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:01 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T07:50:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:01 smithi155 ceph-mon[32688]: osdmap e525: 8 total, 8 up, 8 in 2024-02-10T07:50:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:50:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:01 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T07:50:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:50:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:50:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:50:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[39923]: osdmap e526: 8 total, 8 up, 8 in 2024-02-10T07:50:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:50:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:50:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[39923]: pgmap v798: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:50:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[29548]: osdmap e526: 8 total, 8 up, 8 in 2024-02-10T07:50:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:50:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:50:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:02 smithi070 ceph-mon[29548]: pgmap v798: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:02 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T07:50:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:02 smithi155 ceph-mon[32688]: osdmap e526: 8 total, 8 up, 8 in 2024-02-10T07:50:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:50:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:02 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T07:50:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:02 smithi155 ceph-mon[32688]: pgmap v798: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:50:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[39923]: osdmap e527: 8 total, 8 up, 8 in 2024-02-10T07:50:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:50:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:50:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-102"}]: dispatch 2024-02-10T07:50:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[39923]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-102"}]: dispatch 2024-02-10T07:50:03.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:50:03.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[29548]: osdmap e527: 8 total, 8 up, 8 in 2024-02-10T07:50:03.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:50:03.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:50:03.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-102"}]: dispatch 2024-02-10T07:50:03.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:03 smithi070 ceph-mon[29548]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-102"}]: dispatch 2024-02-10T07:50:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T07:50:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:03 smithi155 ceph-mon[32688]: osdmap e527: 8 total, 8 up, 8 in 2024-02-10T07:50:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:50:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-6"}]: dispatch 2024-02-10T07:50:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3274948648' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-102"}]: dispatch 2024-02-10T07:50:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:03 smithi155 ceph-mon[32688]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-6", "tierpool": "test-rados-api-smithi070-77569-102"}]: dispatch 2024-02-10T07:50:04.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:04 smithi070 ceph-mon[29548]: osdmap e528: 8 total, 8 up, 8 in 2024-02-10T07:50:04.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:04 smithi070 ceph-mon[29548]: pgmap v801: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:50:05.021 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:04 smithi070 ceph-mon[39923]: osdmap e528: 8 total, 8 up, 8 in 2024-02-10T07:50:05.021 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:04 smithi070 ceph-mon[39923]: pgmap v801: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:50:05.021 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:04 smithi155 ceph-mon[32688]: osdmap e528: 8 total, 8 up, 8 in 2024-02-10T07:50:05.021 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:04 smithi155 ceph-mon[32688]: pgmap v801: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:50:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[29548]: osdmap e529: 8 total, 8 up, 8 in 2024-02-10T07:50:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[39923]: osdmap e529: 8 total, 8 up, 8 in 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:05 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:05 smithi155 ceph-mon[32688]: osdmap e529: 8 total, 8 up, 8 in 2024-02-10T07:50:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:05 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:05 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:06.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:05 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[39923]: osdmap e530: 8 total, 8 up, 8 in 2024-02-10T07:50:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[39923]: pgmap v804: 292 pgs: 292 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:06.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:06.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[29548]: osdmap e530: 8 total, 8 up, 8 in 2024-02-10T07:50:06.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:06.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:06 smithi070 ceph-mon[29548]: pgmap v804: 292 pgs: 292 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:06 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:06 smithi155 ceph-mon[32688]: osdmap e530: 8 total, 8 up, 8 in 2024-02-10T07:50:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:06 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:06 smithi155 ceph-mon[32688]: pgmap v804: 292 pgs: 292 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:07.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:07 smithi155 ceph-mon[32688]: osdmap e531: 8 total, 8 up, 8 in 2024-02-10T07:50:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:07 smithi070 ceph-mon[29548]: osdmap e531: 8 total, 8 up, 8 in 2024-02-10T07:50:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:07 smithi070 ceph-mon[39923]: osdmap e531: 8 total, 8 up, 8 in 2024-02-10T07:50:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:08 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:08 smithi070 ceph-mon[29548]: osdmap e532: 8 total, 8 up, 8 in 2024-02-10T07:50:08.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:08 smithi070 ceph-mon[29548]: pgmap v807: 300 pgs: 8 unknown, 292 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:08 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:08 smithi070 ceph-mon[39923]: osdmap e532: 8 total, 8 up, 8 in 2024-02-10T07:50:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:08 smithi070 ceph-mon[39923]: pgmap v807: 300 pgs: 8 unknown, 292 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:08 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi070-77569-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:08 smithi155 ceph-mon[32688]: osdmap e532: 8 total, 8 up, 8 in 2024-02-10T07:50:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:08 smithi155 ceph-mon[32688]: pgmap v807: 300 pgs: 8 unknown, 292 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:09.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:50:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:50:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:09 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:09 smithi070 ceph-mon[29548]: osdmap e533: 8 total, 8 up, 8 in 2024-02-10T07:50:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:09 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:09 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:09 smithi070 ceph-mon[39923]: osdmap e533: 8 total, 8 up, 8 in 2024-02-10T07:50:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:09 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:09 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:09 smithi155 ceph-mon[32688]: osdmap e533: 8 total, 8 up, 8 in 2024-02-10T07:50:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:09 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[29548]: osdmap e534: 8 total, 8 up, 8 in 2024-02-10T07:50:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:11.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[29548]: pgmap v810: 332 pgs: 32 unknown, 1 creating+activating, 299 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:11.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:11.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:11.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[29548]: osdmap e535: 8 total, 8 up, 8 in 2024-02-10T07:50:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[39923]: osdmap e534: 8 total, 8 up, 8 in 2024-02-10T07:50:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:11.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:11.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[39923]: pgmap v810: 332 pgs: 32 unknown, 1 creating+activating, 299 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:11.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:11.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:11.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:10 smithi070 ceph-mon[39923]: osdmap e535: 8 total, 8 up, 8 in 2024-02-10T07:50:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:10 smithi155 ceph-mon[32688]: osdmap e534: 8 total, 8 up, 8 in 2024-02-10T07:50:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:10 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:10 smithi155 ceph-mon[32688]: pgmap v810: 332 pgs: 32 unknown, 1 creating+activating, 299 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:10 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:11.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:10 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:11.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:10 smithi155 ceph-mon[32688]: osdmap e535: 8 total, 8 up, 8 in 2024-02-10T07:50:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:50:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:50:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[29548]: osdmap e536: 8 total, 8 up, 8 in 2024-02-10T07:50:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[29548]: pgmap v813: 364 pgs: 64 unknown, 1 creating+activating, 299 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[39923]: osdmap e536: 8 total, 8 up, 8 in 2024-02-10T07:50:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:12 smithi070 ceph-mon[39923]: pgmap v813: 364 pgs: 64 unknown, 1 creating+activating, 299 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:12 smithi155 ceph-mon[32688]: osdmap e536: 8 total, 8 up, 8 in 2024-02-10T07:50:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:12 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:12 smithi155 ceph-mon[32688]: pgmap v813: 364 pgs: 64 unknown, 1 creating+activating, 299 active+clean; 582 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[39923]: osdmap e537: 8 total, 8 up, 8 in 2024-02-10T07:50:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[29548]: osdmap e537: 8 total, 8 up, 8 in 2024-02-10T07:50:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:13 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:13 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:13 smithi155 ceph-mon[32688]: osdmap e537: 8 total, 8 up, 8 in 2024-02-10T07:50:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:13 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:14.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[39923]: osdmap e538: 8 total, 8 up, 8 in 2024-02-10T07:50:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:14.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[39923]: pgmap v816: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:14.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[29548]: osdmap e538: 8 total, 8 up, 8 in 2024-02-10T07:50:14.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:14.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:14.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:14 smithi070 ceph-mon[29548]: pgmap v816: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:14 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:14 smithi155 ceph-mon[32688]: osdmap e538: 8 total, 8 up, 8 in 2024-02-10T07:50:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:14 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:15.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:14 smithi155 ceph-mon[32688]: pgmap v816: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[39923]: osdmap e539: 8 total, 8 up, 8 in 2024-02-10T07:50:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[29548]: osdmap e539: 8 total, 8 up, 8 in 2024-02-10T07:50:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:15.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:15 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:15 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:15 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-107", "overlaypool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:15 smithi155 ceph-mon[32688]: osdmap e539: 8 total, 8 up, 8 in 2024-02-10T07:50:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:15 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]': finished 2024-02-10T07:50:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[39923]: osdmap e540: 8 total, 8 up, 8 in 2024-02-10T07:50:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]: dispatch 2024-02-10T07:50:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]: dispatch 2024-02-10T07:50:16.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[39923]: pgmap v819: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]': finished 2024-02-10T07:50:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[29548]: osdmap e540: 8 total, 8 up, 8 in 2024-02-10T07:50:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]: dispatch 2024-02-10T07:50:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]: dispatch 2024-02-10T07:50:16.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:16 smithi070 ceph-mon[29548]: pgmap v819: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:16 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:16 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-107-cache", "mode": "writeback"}]': finished 2024-02-10T07:50:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:16 smithi155 ceph-mon[32688]: osdmap e540: 8 total, 8 up, 8 in 2024-02-10T07:50:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]: dispatch 2024-02-10T07:50:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:16 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]: dispatch 2024-02-10T07:50:17.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:16 smithi155 ceph-mon[32688]: pgmap v819: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:50:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]': finished 2024-02-10T07:50:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[39923]: osdmap e541: 8 total, 8 up, 8 in 2024-02-10T07:50:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]': finished 2024-02-10T07:50:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[29548]: osdmap e541: 8 total, 8 up, 8 in 2024-02-10T07:50:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:17.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:17 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:17 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-107"}]': finished 2024-02-10T07:50:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:17 smithi155 ceph-mon[32688]: osdmap e541: 8 total, 8 up, 8 in 2024-02-10T07:50:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:17 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3735403231' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:17 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]: dispatch 2024-02-10T07:50:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[29548]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[29548]: osdmap e542: 8 total, 8 up, 8 in 2024-02-10T07:50:18.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[29548]: pgmap v822: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[39923]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[39923]: osdmap e542: 8 total, 8 up, 8 in 2024-02-10T07:50:18.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:18 smithi070 ceph-mon[39923]: pgmap v822: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:18 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:18 smithi155 ceph-mon[32688]: from='client.36329 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-107", "tierpool": "test-rados-api-smithi070-77569-107-cache"}]': finished 2024-02-10T07:50:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:18 smithi155 ceph-mon[32688]: osdmap e542: 8 total, 8 up, 8 in 2024-02-10T07:50:19.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:18 smithi155 ceph-mon[32688]: pgmap v822: 364 pgs: 364 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:19.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:50:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:50:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:19 smithi155 ceph-mon[32688]: osdmap e543: 8 total, 8 up, 8 in 2024-02-10T07:50:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:19 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:20.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:19 smithi070 ceph-mon[39923]: osdmap e543: 8 total, 8 up, 8 in 2024-02-10T07:50:20.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:19 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:20.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:19 smithi070 ceph-mon[29548]: osdmap e543: 8 total, 8 up, 8 in 2024-02-10T07:50:20.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:19 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:20 smithi155 ceph-mon[32688]: osdmap e544: 8 total, 8 up, 8 in 2024-02-10T07:50:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:21.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:20 smithi155 ceph-mon[32688]: pgmap v825: 300 pgs: 300 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:21.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[39923]: osdmap e544: 8 total, 8 up, 8 in 2024-02-10T07:50:21.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:21.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:21.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:21.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[39923]: pgmap v825: 300 pgs: 300 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:21.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[29548]: osdmap e544: 8 total, 8 up, 8 in 2024-02-10T07:50:21.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:21.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:21.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:21.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:20 smithi070 ceph-mon[29548]: pgmap v825: 300 pgs: 300 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:21 smithi155 ceph-mon[32688]: osdmap e545: 8 total, 8 up, 8 in 2024-02-10T07:50:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:50:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:50:22.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:22.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:21 smithi070 ceph-mon[29548]: osdmap e545: 8 total, 8 up, 8 in 2024-02-10T07:50:22.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:22.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:22.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:21 smithi070 ceph-mon[39923]: osdmap e545: 8 total, 8 up, 8 in 2024-02-10T07:50:22.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:22 smithi155 ceph-mon[32688]: osdmap e546: 8 total, 8 up, 8 in 2024-02-10T07:50:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:22 smithi155 ceph-mon[32688]: pgmap v828: 300 pgs: 300 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:23 smithi155 ceph-mon[32688]: osdmap e547: 8 total, 8 up, 8 in 2024-02-10T07:50:23.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[39923]: osdmap e546: 8 total, 8 up, 8 in 2024-02-10T07:50:23.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[39923]: pgmap v828: 300 pgs: 300 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:23.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:23.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[39923]: osdmap e547: 8 total, 8 up, 8 in 2024-02-10T07:50:23.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[29548]: osdmap e546: 8 total, 8 up, 8 in 2024-02-10T07:50:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[29548]: pgmap v828: 300 pgs: 300 active+clean; 582 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:22 smithi070 ceph-mon[29548]: osdmap e547: 8 total, 8 up, 8 in 2024-02-10T07:50:25.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:24 smithi070 ceph-mon[39923]: osdmap e548: 8 total, 8 up, 8 in 2024-02-10T07:50:25.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:25.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:24 smithi070 ceph-mon[39923]: pgmap v831: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:24 smithi070 ceph-mon[29548]: osdmap e548: 8 total, 8 up, 8 in 2024-02-10T07:50:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:25.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:24 smithi070 ceph-mon[29548]: pgmap v831: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:24 smithi155 ceph-mon[32688]: osdmap e548: 8 total, 8 up, 8 in 2024-02-10T07:50:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:25.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:24 smithi155 ceph-mon[32688]: pgmap v831: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:26.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[29548]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[29548]: osdmap e549: 8 total, 8 up, 8 in 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[39923]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[39923]: osdmap e549: 8 total, 8 up, 8 in 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:25 smithi155 ceph-mon[32688]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:25 smithi155 ceph-mon[32688]: osdmap e549: 8 total, 8 up, 8 in 2024-02-10T07:50:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[29548]: osdmap e550: 8 total, 8 up, 8 in 2024-02-10T07:50:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-109", "overlaypool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[29548]: pgmap v834: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[39923]: osdmap e550: 8 total, 8 up, 8 in 2024-02-10T07:50:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-109", "overlaypool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:26 smithi070 ceph-mon[39923]: pgmap v834: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:26 smithi155 ceph-mon[32688]: osdmap e550: 8 total, 8 up, 8 in 2024-02-10T07:50:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-109", "overlaypool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:26 smithi155 ceph-mon[32688]: pgmap v834: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:28.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-109", "overlaypool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:28.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[29548]: osdmap e551: 8 total, 8 up, 8 in 2024-02-10T07:50:28.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-109-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:28.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:50:28.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-109", "overlaypool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:28.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[39923]: osdmap e551: 8 total, 8 up, 8 in 2024-02-10T07:50:28.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-109-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:28.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:50:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-109", "overlaypool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:27 smithi155 ceph-mon[32688]: osdmap e551: 8 total, 8 up, 8 in 2024-02-10T07:50:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-109-cache", "mode": "writeback"}]: dispatch 2024-02-10T07:50:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:50:29.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:29.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-109-cache", "mode": "writeback"}]': finished 2024-02-10T07:50:29.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: osdmap e552: 8 total, 8 up, 8 in 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: pgmap v837: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: osdmap e553: 8 total, 8 up, 8 in 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-109-cache", "mode": "writeback"}]': finished 2024-02-10T07:50:29.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: osdmap e552: 8 total, 8 up, 8 in 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: pgmap v837: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: osdmap e553: 8 total, 8 up, 8 in 2024-02-10T07:50:29.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-109-cache", "mode": "writeback"}]': finished 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: osdmap e552: 8 total, 8 up, 8 in 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: pgmap v837: 340 pgs: 32 creating+peering, 308 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:50:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: osdmap e553: 8 total, 8 up, 8 in 2024-02-10T07:50:29.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:50:29.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:50:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:50:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:50:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[29548]: osdmap e554: 8 total, 8 up, 8 in 2024-02-10T07:50:30.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:50:30.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:30.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:50:30.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[39923]: osdmap e554: 8 total, 8 up, 8 in 2024-02-10T07:50:30.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:50:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:29 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:50:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:29 smithi155 ceph-mon[32688]: osdmap e554: 8 total, 8 up, 8 in 2024-02-10T07:50:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:50:31.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[29548]: pgmap v840: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:31.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:31.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:50:31.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[29548]: osdmap e555: 8 total, 8 up, 8 in 2024-02-10T07:50:31.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T07:50:31.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[39923]: pgmap v840: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:31.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:31.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:50:31.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[39923]: osdmap e555: 8 total, 8 up, 8 in 2024-02-10T07:50:31.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T07:50:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:30 smithi155 ceph-mon[32688]: pgmap v840: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:30 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:50:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:30 smithi155 ceph-mon[32688]: osdmap e555: 8 total, 8 up, 8 in 2024-02-10T07:50:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T07:50:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:50:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:50:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T07:50:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[29548]: osdmap e556: 8 total, 8 up, 8 in 2024-02-10T07:50:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[29548]: pgmap v843: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T07:50:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[39923]: osdmap e556: 8 total, 8 up, 8 in 2024-02-10T07:50:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[39923]: pgmap v843: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:32 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T07:50:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:32 smithi155 ceph-mon[32688]: osdmap e556: 8 total, 8 up, 8 in 2024-02-10T07:50:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:32 smithi155 ceph-mon[32688]: pgmap v843: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:32 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:34.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:34.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:33 smithi070 ceph-mon[39923]: osdmap e557: 8 total, 8 up, 8 in 2024-02-10T07:50:34.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:33 smithi070 ceph-mon[29548]: osdmap e557: 8 total, 8 up, 8 in 2024-02-10T07:50:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:33 smithi155 ceph-mon[32688]: osdmap e557: 8 total, 8 up, 8 in 2024-02-10T07:50:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]: dispatch 2024-02-10T07:50:35.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:35.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:34 smithi070 ceph-mon[39923]: osdmap e558: 8 total, 8 up, 8 in 2024-02-10T07:50:35.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:34 smithi070 ceph-mon[39923]: pgmap v846: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:50:35.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:34 smithi070 ceph-mon[29548]: osdmap e558: 8 total, 8 up, 8 in 2024-02-10T07:50:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:34 smithi070 ceph-mon[29548]: pgmap v846: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:50:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-109", "tierpool": "test-rados-api-smithi070-77569-109-cache"}]': finished 2024-02-10T07:50:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:34 smithi155 ceph-mon[32688]: osdmap e558: 8 total, 8 up, 8 in 2024-02-10T07:50:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:34 smithi155 ceph-mon[32688]: pgmap v846: 340 pgs: 340 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T07:50:36.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:35 smithi070 ceph-mon[29548]: osdmap e559: 8 total, 8 up, 8 in 2024-02-10T07:50:36.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:35 smithi070 ceph-mon[39923]: osdmap e559: 8 total, 8 up, 8 in 2024-02-10T07:50:36.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:35 smithi155 ceph-mon[32688]: osdmap e559: 8 total, 8 up, 8 in 2024-02-10T07:50:37.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:36 smithi155 ceph-mon[32688]: osdmap e560: 8 total, 8 up, 8 in 2024-02-10T07:50:37.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:37.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:36 smithi155 ceph-mon[32688]: pgmap v849: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:37.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:37.157 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:36 smithi155 ceph-mon[32688]: osdmap e561: 8 total, 8 up, 8 in 2024-02-10T07:50:37.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[29548]: osdmap e560: 8 total, 8 up, 8 in 2024-02-10T07:50:37.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:37.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[29548]: pgmap v849: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:37.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:37.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[29548]: osdmap e561: 8 total, 8 up, 8 in 2024-02-10T07:50:37.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[39923]: osdmap e560: 8 total, 8 up, 8 in 2024-02-10T07:50:37.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:37.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[39923]: pgmap v849: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:37.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:37.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:36 smithi070 ceph-mon[39923]: osdmap e561: 8 total, 8 up, 8 in 2024-02-10T07:50:38.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:38.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:38.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[29548]: osdmap e562: 8 total, 8 up, 8 in 2024-02-10T07:50:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:37 smithi070 ceph-mon[39923]: osdmap e562: 8 total, 8 up, 8 in 2024-02-10T07:50:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]: dispatch 2024-02-10T07:50:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:37 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/943968275' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-109"}]': finished 2024-02-10T07:50:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:37 smithi155 ceph-mon[32688]: osdmap e562: 8 total, 8 up, 8 in 2024-02-10T07:50:39.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[29548]: pgmap v852: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:39.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[29548]: osdmap e563: 8 total, 8 up, 8 in 2024-02-10T07:50:39.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:39.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:39.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[39923]: pgmap v852: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:39.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[39923]: osdmap e563: 8 total, 8 up, 8 in 2024-02-10T07:50:39.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:39.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:38 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:39.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:38 smithi155 ceph-mon[32688]: pgmap v852: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:39.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:38 smithi155 ceph-mon[32688]: osdmap e563: 8 total, 8 up, 8 in 2024-02-10T07:50:39.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:39.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:38 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:39.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:50:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:50:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[29548]: osdmap e564: 8 total, 8 up, 8 in 2024-02-10T07:50:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:40.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[39923]: osdmap e564: 8 total, 8 up, 8 in 2024-02-10T07:50:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:39 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:39 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:39 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:39 smithi155 ceph-mon[32688]: osdmap e564: 8 total, 8 up, 8 in 2024-02-10T07:50:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3916598983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:39 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]: dispatch 2024-02-10T07:50:41.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[29548]: pgmap v855: 292 pgs: 292 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:50:41.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:41.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[29548]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:41.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[29548]: osdmap e565: 8 total, 8 up, 8 in 2024-02-10T07:50:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[39923]: pgmap v855: 292 pgs: 292 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:50:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[39923]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:40 smithi070 ceph-mon[39923]: osdmap e565: 8 total, 8 up, 8 in 2024-02-10T07:50:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:40 smithi155 ceph-mon[32688]: pgmap v855: 292 pgs: 292 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:50:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:40 smithi155 ceph-mon[32688]: from='client.36323 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi070-77569-104"}]': finished 2024-02-10T07:50:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:40 smithi155 ceph-mon[32688]: osdmap e565: 8 total, 8 up, 8 in 2024-02-10T07:50:42.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:50:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:50:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:50:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:42.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T07:50:43.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:43.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[29548]: osdmap e566: 8 total, 8 up, 8 in 2024-02-10T07:50:43.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:43.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[29548]: pgmap v858: 292 pgs: 292 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:50:43.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:43.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:43.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[39923]: osdmap e566: 8 total, 8 up, 8 in 2024-02-10T07:50:43.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:43.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[39923]: pgmap v858: 292 pgs: 292 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:50:43.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:43 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:43.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi070-77569-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T07:50:43.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:43 smithi155 ceph-mon[32688]: osdmap e566: 8 total, 8 up, 8 in 2024-02-10T07:50:43.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:43.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:43 smithi155 ceph-mon[32688]: pgmap v858: 292 pgs: 292 active+clean; 582 KiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T07:50:43.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:43 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:44.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:44 smithi070 ceph-mon[29548]: osdmap e567: 8 total, 8 up, 8 in 2024-02-10T07:50:44.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:44.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:44 smithi070 ceph-mon[29548]: osdmap e568: 8 total, 8 up, 8 in 2024-02-10T07:50:44.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:44 smithi070 ceph-mon[39923]: osdmap e567: 8 total, 8 up, 8 in 2024-02-10T07:50:44.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:44.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:44 smithi070 ceph-mon[39923]: osdmap e568: 8 total, 8 up, 8 in 2024-02-10T07:50:44.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:44 smithi155 ceph-mon[32688]: osdmap e567: 8 total, 8 up, 8 in 2024-02-10T07:50:44.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi070-77569-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:44.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:44 smithi155 ceph-mon[32688]: osdmap e568: 8 total, 8 up, 8 in 2024-02-10T07:50:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:45 smithi070 ceph-mon[29548]: pgmap v861: 300 pgs: 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:45.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:45 smithi070 ceph-mon[29548]: osdmap e569: 8 total, 8 up, 8 in 2024-02-10T07:50:45.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:45.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:45 smithi070 ceph-mon[39923]: pgmap v861: 300 pgs: 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:45.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:45 smithi070 ceph-mon[39923]: osdmap e569: 8 total, 8 up, 8 in 2024-02-10T07:50:45.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:45 smithi155 ceph-mon[32688]: pgmap v861: 300 pgs: 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:45 smithi155 ceph-mon[32688]: osdmap e569: 8 total, 8 up, 8 in 2024-02-10T07:50:45.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:47.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:47.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[29548]: osdmap e570: 8 total, 8 up, 8 in 2024-02-10T07:50:47.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:50:47.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[29548]: pgmap v864: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:47.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:47.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[39923]: osdmap e570: 8 total, 8 up, 8 in 2024-02-10T07:50:47.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:50:47.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:46 smithi070 ceph-mon[39923]: pgmap v864: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:46 smithi155 ceph-mon[32688]: osdmap e570: 8 total, 8 up, 8 in 2024-02-10T07:50:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:50:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:46 smithi155 ceph-mon[32688]: pgmap v864: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:50:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:50:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[29548]: osdmap e571: 8 total, 8 up, 8 in 2024-02-10T07:50:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-112"}]: dispatch 2024-02-10T07:50:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:50:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[39923]: osdmap e571: 8 total, 8 up, 8 in 2024-02-10T07:50:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-112"}]: dispatch 2024-02-10T07:50:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:47 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:50:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:47 smithi155 ceph-mon[32688]: osdmap e571: 8 total, 8 up, 8 in 2024-02-10T07:50:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-112"}]: dispatch 2024-02-10T07:50:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:47 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:49.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-112"}]': finished 2024-02-10T07:50:49.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[39923]: osdmap e572: 8 total, 8 up, 8 in 2024-02-10T07:50:49.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:49.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[39923]: pgmap v867: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:49.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-112"}]': finished 2024-02-10T07:50:49.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[29548]: osdmap e572: 8 total, 8 up, 8 in 2024-02-10T07:50:49.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:49.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:48 smithi070 ceph-mon[29548]: pgmap v867: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-112"}]': finished 2024-02-10T07:50:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:48 smithi155 ceph-mon[32688]: osdmap e572: 8 total, 8 up, 8 in 2024-02-10T07:50:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:48 smithi155 ceph-mon[32688]: pgmap v867: 332 pgs: 32 unknown, 8 creating+peering, 292 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:49.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:50:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:50:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[29548]: osdmap e573: 8 total, 8 up, 8 in 2024-02-10T07:50:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112"}]: dispatch 2024-02-10T07:50:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[39923]: osdmap e573: 8 total, 8 up, 8 in 2024-02-10T07:50:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112"}]: dispatch 2024-02-10T07:50:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:49 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:49 smithi155 ceph-mon[32688]: osdmap e573: 8 total, 8 up, 8 in 2024-02-10T07:50:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112"}]: dispatch 2024-02-10T07:50:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:49 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:50:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112"}]': finished 2024-02-10T07:50:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:50 smithi070 ceph-mon[29548]: osdmap e574: 8 total, 8 up, 8 in 2024-02-10T07:50:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:50 smithi070 ceph-mon[29548]: pgmap v870: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:50:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112"}]': finished 2024-02-10T07:50:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:50 smithi070 ceph-mon[39923]: osdmap e574: 8 total, 8 up, 8 in 2024-02-10T07:50:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:50 smithi070 ceph-mon[39923]: pgmap v870: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:50:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-112"}]': finished 2024-02-10T07:50:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:50 smithi155 ceph-mon[32688]: osdmap e574: 8 total, 8 up, 8 in 2024-02-10T07:50:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:50 smithi155 ceph-mon[32688]: pgmap v870: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:50:52.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:51 smithi070 ceph-mon[29548]: osdmap e575: 8 total, 8 up, 8 in 2024-02-10T07:50:52.142 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:51 smithi070 ceph-mon[39923]: osdmap e575: 8 total, 8 up, 8 in 2024-02-10T07:50:52.143 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:50:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:50:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:51 smithi155 ceph-mon[32688]: osdmap e575: 8 total, 8 up, 8 in 2024-02-10T07:50:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[29548]: osdmap e576: 8 total, 8 up, 8 in 2024-02-10T07:50:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[29548]: pgmap v873: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:50:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[39923]: osdmap e576: 8 total, 8 up, 8 in 2024-02-10T07:50:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[39923]: pgmap v873: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:50:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:52 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:52 smithi155 ceph-mon[32688]: osdmap e576: 8 total, 8 up, 8 in 2024-02-10T07:50:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:50:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:52 smithi155 ceph-mon[32688]: pgmap v873: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:50:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:52 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:53 smithi070 ceph-mon[29548]: osdmap e577: 8 total, 8 up, 8 in 2024-02-10T07:50:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:50:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:53 smithi070 ceph-mon[39923]: osdmap e577: 8 total, 8 up, 8 in 2024-02-10T07:50:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:50:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:50:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:53 smithi155 ceph-mon[32688]: osdmap e577: 8 total, 8 up, 8 in 2024-02-10T07:50:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:50:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:50:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[29548]: osdmap e578: 8 total, 8 up, 8 in 2024-02-10T07:50:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-114"}]: dispatch 2024-02-10T07:50:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[29548]: pgmap v876: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:50:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:50:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[39923]: osdmap e578: 8 total, 8 up, 8 in 2024-02-10T07:50:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-114"}]: dispatch 2024-02-10T07:50:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:54 smithi070 ceph-mon[39923]: pgmap v876: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:50:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:50:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:54 smithi155 ceph-mon[32688]: osdmap e578: 8 total, 8 up, 8 in 2024-02-10T07:50:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-114"}]: dispatch 2024-02-10T07:50:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:54 smithi155 ceph-mon[32688]: pgmap v876: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:50:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-114"}]': finished 2024-02-10T07:50:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[29548]: osdmap e579: 8 total, 8 up, 8 in 2024-02-10T07:50:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-114", "mode": "writeback"}]: dispatch 2024-02-10T07:50:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-114"}]': finished 2024-02-10T07:50:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[39923]: osdmap e579: 8 total, 8 up, 8 in 2024-02-10T07:50:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-114", "mode": "writeback"}]: dispatch 2024-02-10T07:50:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:55 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-114"}]': finished 2024-02-10T07:50:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:55 smithi155 ceph-mon[32688]: osdmap e579: 8 total, 8 up, 8 in 2024-02-10T07:50:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-114", "mode": "writeback"}]: dispatch 2024-02-10T07:50:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:55 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:50:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-114", "mode": "writeback"}]': finished 2024-02-10T07:50:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[29548]: osdmap e580: 8 total, 8 up, 8 in 2024-02-10T07:50:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[29548]: pgmap v879: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:50:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-114", "mode": "writeback"}]': finished 2024-02-10T07:50:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[39923]: osdmap e580: 8 total, 8 up, 8 in 2024-02-10T07:50:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:56 smithi070 ceph-mon[39923]: pgmap v879: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:50:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:56 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:50:57.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-114", "mode": "writeback"}]': finished 2024-02-10T07:50:57.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:56 smithi155 ceph-mon[32688]: osdmap e580: 8 total, 8 up, 8 in 2024-02-10T07:50:57.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:50:57.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:56 smithi155 ceph-mon[32688]: pgmap v879: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:50:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[39923]: osdmap e581: 8 total, 8 up, 8 in 2024-02-10T07:50:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114"}]: dispatch 2024-02-10T07:50:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[29548]: osdmap e581: 8 total, 8 up, 8 in 2024-02-10T07:50:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114"}]: dispatch 2024-02-10T07:50:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:57 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:50:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:57 smithi155 ceph-mon[32688]: osdmap e581: 8 total, 8 up, 8 in 2024-02-10T07:50:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114"}]: dispatch 2024-02-10T07:50:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:57 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:50:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114"}]': finished 2024-02-10T07:50:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[29548]: osdmap e582: 8 total, 8 up, 8 in 2024-02-10T07:50:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[29548]: pgmap v882: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:59.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114"}]': finished 2024-02-10T07:50:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[39923]: osdmap e582: 8 total, 8 up, 8 in 2024-02-10T07:50:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:58 smithi070 ceph-mon[39923]: pgmap v882: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:58 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:50:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-114"}]': finished 2024-02-10T07:50:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:58 smithi155 ceph-mon[32688]: osdmap e582: 8 total, 8 up, 8 in 2024-02-10T07:50:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:58 smithi155 ceph-mon[32688]: pgmap v882: 332 pgs: 332 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:50:59.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:50:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:51:00.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:59 smithi070 ceph-mon[39923]: osdmap e583: 8 total, 8 up, 8 in 2024-02-10T07:51:00.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:50:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:59 smithi070 ceph-mon[29548]: osdmap e583: 8 total, 8 up, 8 in 2024-02-10T07:51:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:50:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:59 smithi155 ceph-mon[32688]: osdmap e583: 8 total, 8 up, 8 in 2024-02-10T07:51:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:50:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:00 smithi070 ceph-mon[39923]: osdmap e584: 8 total, 8 up, 8 in 2024-02-10T07:51:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:00 smithi070 ceph-mon[39923]: pgmap v885: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:00 smithi070 ceph-mon[29548]: osdmap e584: 8 total, 8 up, 8 in 2024-02-10T07:51:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:00 smithi070 ceph-mon[29548]: pgmap v885: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:00 smithi155 ceph-mon[32688]: osdmap e584: 8 total, 8 up, 8 in 2024-02-10T07:51:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:00 smithi155 ceph-mon[32688]: pgmap v885: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:02.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:02.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:01 smithi070 ceph-mon[29548]: osdmap e585: 8 total, 8 up, 8 in 2024-02-10T07:51:02.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:02.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:01 smithi070 ceph-mon[39923]: osdmap e585: 8 total, 8 up, 8 in 2024-02-10T07:51:02.242 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:51:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:51:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:02.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:01 smithi155 ceph-mon[32688]: osdmap e585: 8 total, 8 up, 8 in 2024-02-10T07:51:03.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[39923]: osdmap e586: 8 total, 8 up, 8 in 2024-02-10T07:51:03.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:03.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[39923]: pgmap v888: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:03.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:51:03.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[29548]: osdmap e586: 8 total, 8 up, 8 in 2024-02-10T07:51:03.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:03.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[29548]: pgmap v888: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:03.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:02 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:51:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:02 smithi155 ceph-mon[32688]: osdmap e586: 8 total, 8 up, 8 in 2024-02-10T07:51:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:02 smithi155 ceph-mon[32688]: pgmap v888: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:02 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:51:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:03 smithi070 ceph-mon[29548]: osdmap e587: 8 total, 8 up, 8 in 2024-02-10T07:51:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-116"}]: dispatch 2024-02-10T07:51:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:03 smithi070 ceph-mon[39923]: osdmap e587: 8 total, 8 up, 8 in 2024-02-10T07:51:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-116"}]: dispatch 2024-02-10T07:51:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:03 smithi155 ceph-mon[32688]: osdmap e587: 8 total, 8 up, 8 in 2024-02-10T07:51:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-116"}]: dispatch 2024-02-10T07:51:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-116"}]': finished 2024-02-10T07:51:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[29548]: osdmap e588: 8 total, 8 up, 8 in 2024-02-10T07:51:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-116", "mode": "writeback"}]: dispatch 2024-02-10T07:51:05.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[29548]: pgmap v891: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-116", "mode": "writeback"}]': finished 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-116"}]': finished 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[39923]: osdmap e588: 8 total, 8 up, 8 in 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-116", "mode": "writeback"}]: dispatch 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[39923]: pgmap v891: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:05.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-116", "mode": "writeback"}]': finished 2024-02-10T07:51:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-116"}]': finished 2024-02-10T07:51:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:04 smithi155 ceph-mon[32688]: osdmap e588: 8 total, 8 up, 8 in 2024-02-10T07:51:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-116", "mode": "writeback"}]: dispatch 2024-02-10T07:51:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:04 smithi155 ceph-mon[32688]: pgmap v891: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:05.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:04 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:05.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-116", "mode": "writeback"}]': finished 2024-02-10T07:51:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[39923]: osdmap e589: 8 total, 8 up, 8 in 2024-02-10T07:51:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "314.7"}]: dispatch 2024-02-10T07:51:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[39923]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "314.7"}]: dispatch 2024-02-10T07:51:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[39923]: 314.7 scrub starts 2024-02-10T07:51:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[39923]: 314.7 scrub ok 2024-02-10T07:51:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[29548]: osdmap e589: 8 total, 8 up, 8 in 2024-02-10T07:51:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "314.7"}]: dispatch 2024-02-10T07:51:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[29548]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "314.7"}]: dispatch 2024-02-10T07:51:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[29548]: 314.7 scrub starts 2024-02-10T07:51:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:05 smithi070 ceph-mon[29548]: 314.7 scrub ok 2024-02-10T07:51:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:05 smithi155 ceph-mon[32688]: osdmap e589: 8 total, 8 up, 8 in 2024-02-10T07:51:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "314.7"}]: dispatch 2024-02-10T07:51:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:05 smithi155 ceph-mon[32688]: from='mon.0 v1:172.21.15.70:0/1633077837' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "314.7"}]: dispatch 2024-02-10T07:51:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:05 smithi155 ceph-mon[32688]: 314.7 scrub starts 2024-02-10T07:51:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:05 smithi155 ceph-mon[32688]: 314.7 scrub ok 2024-02-10T07:51:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:06 smithi070 ceph-mon[29548]: pgmap v893: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:07.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:06 smithi070 ceph-mon[39923]: pgmap v893: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:07.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:06 smithi155 ceph-mon[32688]: pgmap v893: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:07 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:51:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:07 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:51:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:07 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:51:09.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:08 smithi070 ceph-mon[29548]: pgmap v894: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:51:09.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:08 smithi070 ceph-mon[39923]: pgmap v894: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:51:09.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:08 smithi155 ceph-mon[32688]: pgmap v894: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:51:09.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:51:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:51:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:09 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:10.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:09 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:09 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:11 smithi070 ceph-mon[39923]: pgmap v895: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1015 B/s wr, 3 op/s 2024-02-10T07:51:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:11 smithi070 ceph-mon[29548]: pgmap v895: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1015 B/s wr, 3 op/s 2024-02-10T07:51:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:11 smithi155 ceph-mon[32688]: pgmap v895: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1015 B/s wr, 3 op/s 2024-02-10T07:51:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:51:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:51:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:51:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:13 smithi070 ceph-mon[29548]: pgmap v896: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 889 B/s rd, 1 op/s 2024-02-10T07:51:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:13 smithi070 ceph-mon[39923]: pgmap v896: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 889 B/s rd, 1 op/s 2024-02-10T07:51:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:13 smithi155 ceph-mon[32688]: pgmap v896: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 889 B/s rd, 1 op/s 2024-02-10T07:51:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:15 smithi070 ceph-mon[29548]: pgmap v897: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:15 smithi070 ceph-mon[39923]: pgmap v897: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:15 smithi155 ceph-mon[32688]: pgmap v897: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:17.451 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.451 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.451 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.451 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.452 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.453 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.454 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.455 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.456 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.457 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.458 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:51:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:17 smithi070 ceph-mon[29548]: pgmap v898: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:51:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:17 smithi070 ceph-mon[39923]: pgmap v898: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:51:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:17 smithi155 ceph-mon[32688]: pgmap v898: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:51:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:19 smithi155 ceph-mon[32688]: pgmap v899: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T07:51:19.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:51:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:51:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:19 smithi070 ceph-mon[29548]: pgmap v899: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T07:51:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:19 smithi070 ceph-mon[39923]: pgmap v899: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T07:51:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:21 smithi070 ceph-mon[29548]: pgmap v900: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:51:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:21 smithi070 ceph-mon[29548]: osdmap e590: 8 total, 8 up, 8 in 2024-02-10T07:51:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:21 smithi070 ceph-mon[39923]: pgmap v900: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:51:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:21 smithi070 ceph-mon[39923]: osdmap e590: 8 total, 8 up, 8 in 2024-02-10T07:51:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:21 smithi155 ceph-mon[32688]: pgmap v900: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T07:51:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:21 smithi155 ceph-mon[32688]: osdmap e590: 8 total, 8 up, 8 in 2024-02-10T07:51:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:51:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:51:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[29548]: osdmap e591: 8 total, 8 up, 8 in 2024-02-10T07:51:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116"}]: dispatch 2024-02-10T07:51:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116"}]': finished 2024-02-10T07:51:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[39923]: osdmap e591: 8 total, 8 up, 8 in 2024-02-10T07:51:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116"}]: dispatch 2024-02-10T07:51:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116"}]': finished 2024-02-10T07:51:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:22 smithi155 ceph-mon[32688]: osdmap e591: 8 total, 8 up, 8 in 2024-02-10T07:51:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116"}]: dispatch 2024-02-10T07:51:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:22 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-116"}]': finished 2024-02-10T07:51:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:23 smithi070 ceph-mon[29548]: pgmap v903: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:51:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:23 smithi070 ceph-mon[29548]: osdmap e592: 8 total, 8 up, 8 in 2024-02-10T07:51:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:23 smithi070 ceph-mon[39923]: pgmap v903: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:51:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:23 smithi070 ceph-mon[39923]: osdmap e592: 8 total, 8 up, 8 in 2024-02-10T07:51:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:23 smithi155 ceph-mon[32688]: pgmap v903: 332 pgs: 332 active+clean; 582 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:51:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:23 smithi155 ceph-mon[32688]: osdmap e592: 8 total, 8 up, 8 in 2024-02-10T07:51:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:24 smithi070 ceph-mon[29548]: osdmap e593: 8 total, 8 up, 8 in 2024-02-10T07:51:24.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:24 smithi070 ceph-mon[39923]: osdmap e593: 8 total, 8 up, 8 in 2024-02-10T07:51:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:24 smithi155 ceph-mon[32688]: osdmap e593: 8 total, 8 up, 8 in 2024-02-10T07:51:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[29548]: pgmap v906: 300 pgs: 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T07:51:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[29548]: osdmap e594: 8 total, 8 up, 8 in 2024-02-10T07:51:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:25.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[39923]: pgmap v906: 300 pgs: 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T07:51:25.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[39923]: osdmap e594: 8 total, 8 up, 8 in 2024-02-10T07:51:25.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:25.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:25.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:25 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:25 smithi155 ceph-mon[32688]: pgmap v906: 300 pgs: 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T07:51:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:25 smithi155 ceph-mon[32688]: osdmap e594: 8 total, 8 up, 8 in 2024-02-10T07:51:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:26.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:26.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:25 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:26 smithi070 ceph-mon[29548]: osdmap e595: 8 total, 8 up, 8 in 2024-02-10T07:51:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:26 smithi070 ceph-mon[39923]: osdmap e595: 8 total, 8 up, 8 in 2024-02-10T07:51:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:26 smithi155 ceph-mon[32688]: osdmap e595: 8 total, 8 up, 8 in 2024-02-10T07:51:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[29548]: pgmap v909: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T07:51:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[29548]: osdmap e596: 8 total, 8 up, 8 in 2024-02-10T07:51:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[29548]: osdmap e597: 8 total, 8 up, 8 in 2024-02-10T07:51:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-118"}]: dispatch 2024-02-10T07:51:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[39923]: pgmap v909: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T07:51:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[39923]: osdmap e596: 8 total, 8 up, 8 in 2024-02-10T07:51:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[39923]: osdmap e597: 8 total, 8 up, 8 in 2024-02-10T07:51:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-118"}]: dispatch 2024-02-10T07:51:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:27 smithi155 ceph-mon[32688]: pgmap v909: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T07:51:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:27 smithi155 ceph-mon[32688]: osdmap e596: 8 total, 8 up, 8 in 2024-02-10T07:51:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:27 smithi155 ceph-mon[32688]: osdmap e597: 8 total, 8 up, 8 in 2024-02-10T07:51:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-118"}]: dispatch 2024-02-10T07:51:28.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:28 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:51:28.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-118"}]': finished 2024-02-10T07:51:28.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:28 smithi155 ceph-mon[32688]: osdmap e598: 8 total, 8 up, 8 in 2024-02-10T07:51:28.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-118", "mode": "writeback"}]: dispatch 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-118"}]': finished 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[29548]: osdmap e598: 8 total, 8 up, 8 in 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-118", "mode": "writeback"}]: dispatch 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-118"}]': finished 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[39923]: osdmap e598: 8 total, 8 up, 8 in 2024-02-10T07:51:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-118", "mode": "writeback"}]: dispatch 2024-02-10T07:51:29.584 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:51:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:51:29.866 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:29 smithi155 ceph-mon[32688]: pgmap v912: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:29.866 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:29 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:29.866 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:29 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-118", "mode": "writeback"}]': finished 2024-02-10T07:51:29.867 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:29 smithi155 ceph-mon[32688]: osdmap e599: 8 total, 8 up, 8 in 2024-02-10T07:51:29.978 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[39923]: pgmap v912: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:29.979 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:29.979 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-118", "mode": "writeback"}]': finished 2024-02-10T07:51:29.979 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[39923]: osdmap e599: 8 total, 8 up, 8 in 2024-02-10T07:51:29.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[29548]: pgmap v912: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:29.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:29.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-118", "mode": "writeback"}]': finished 2024-02-10T07:51:29.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:29 smithi070 ceph-mon[29548]: osdmap e599: 8 total, 8 up, 8 in 2024-02-10T07:51:30.903 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:30.904 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:30 smithi070 ceph-mon[29548]: osdmap e600: 8 total, 8 up, 8 in 2024-02-10T07:51:30.904 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:30.904 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:30.904 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:30 smithi070 ceph-mon[39923]: osdmap e600: 8 total, 8 up, 8 in 2024-02-10T07:51:30.904 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:31.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:31.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:30 smithi155 ceph-mon[32688]: osdmap e600: 8 total, 8 up, 8 in 2024-02-10T07:51:31.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:31.918 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:31 smithi155 ceph-mon[32688]: pgmap v915: 332 pgs: 332 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:31.918 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:31 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:31.918 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:31 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:31.918 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:31.918 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:31 smithi155 ceph-mon[32688]: osdmap e601: 8 total, 8 up, 8 in 2024-02-10T07:51:31.918 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118"}]: dispatch 2024-02-10T07:51:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[39923]: pgmap v915: 332 pgs: 332 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[39923]: osdmap e601: 8 total, 8 up, 8 in 2024-02-10T07:51:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118"}]: dispatch 2024-02-10T07:51:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[29548]: pgmap v915: 332 pgs: 332 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[29548]: osdmap e601: 8 total, 8 up, 8 in 2024-02-10T07:51:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118"}]: dispatch 2024-02-10T07:51:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:51:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:51:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:51:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: pgmap v918: 332 pgs: 332 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118"}]': finished 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: osdmap e602: 8 total, 8 up, 8 in 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: pgmap v918: 332 pgs: 332 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118"}]': finished 2024-02-10T07:51:33.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: osdmap e602: 8 total, 8 up, 8 in 2024-02-10T07:51:33.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:51:33.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:51:33.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:33 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: pgmap v918: 332 pgs: 332 active+clean; 582 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-118"}]': finished 2024-02-10T07:51:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: osdmap e602: 8 total, 8 up, 8 in 2024-02-10T07:51:33.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:51:33.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:51:33.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:33 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:34.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:34 smithi070 ceph-mon[29548]: osdmap e603: 8 total, 8 up, 8 in 2024-02-10T07:51:34.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:34 smithi070 ceph-mon[39923]: osdmap e603: 8 total, 8 up, 8 in 2024-02-10T07:51:34.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:34 smithi155 ceph-mon[32688]: osdmap e603: 8 total, 8 up, 8 in 2024-02-10T07:51:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:35 smithi070 ceph-mon[29548]: pgmap v921: 300 pgs: 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:35 smithi070 ceph-mon[29548]: osdmap e604: 8 total, 8 up, 8 in 2024-02-10T07:51:35.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:35.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:35 smithi070 ceph-mon[39923]: pgmap v921: 300 pgs: 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:35.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:35 smithi070 ceph-mon[39923]: osdmap e604: 8 total, 8 up, 8 in 2024-02-10T07:51:35.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:35 smithi155 ceph-mon[32688]: pgmap v921: 300 pgs: 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:35 smithi155 ceph-mon[32688]: osdmap e604: 8 total, 8 up, 8 in 2024-02-10T07:51:35.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:36.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:36.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:36 smithi070 ceph-mon[29548]: osdmap e605: 8 total, 8 up, 8 in 2024-02-10T07:51:36.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:36.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:36.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:36 smithi070 ceph-mon[39923]: osdmap e605: 8 total, 8 up, 8 in 2024-02-10T07:51:36.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:36 smithi155 ceph-mon[32688]: osdmap e605: 8 total, 8 up, 8 in 2024-02-10T07:51:36.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[29548]: pgmap v924: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:37.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[29548]: osdmap e606: 8 total, 8 up, 8 in 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-120"}]: dispatch 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-120"}]': finished 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[29548]: osdmap e607: 8 total, 8 up, 8 in 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-120", "mode": "writeback"}]: dispatch 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[39923]: pgmap v924: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[39923]: osdmap e606: 8 total, 8 up, 8 in 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-120"}]: dispatch 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-120"}]': finished 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[39923]: osdmap e607: 8 total, 8 up, 8 in 2024-02-10T07:51:37.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-120", "mode": "writeback"}]: dispatch 2024-02-10T07:51:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:37 smithi155 ceph-mon[32688]: pgmap v924: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:51:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:37 smithi155 ceph-mon[32688]: osdmap e606: 8 total, 8 up, 8 in 2024-02-10T07:51:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-120"}]: dispatch 2024-02-10T07:51:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-120"}]': finished 2024-02-10T07:51:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:37 smithi155 ceph-mon[32688]: osdmap e607: 8 total, 8 up, 8 in 2024-02-10T07:51:37.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-120", "mode": "writeback"}]: dispatch 2024-02-10T07:51:38.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:38 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:38.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-120", "mode": "writeback"}]': finished 2024-02-10T07:51:38.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:38 smithi070 ceph-mon[29548]: osdmap e608: 8 total, 8 up, 8 in 2024-02-10T07:51:38.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:38 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:38.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-120", "mode": "writeback"}]': finished 2024-02-10T07:51:38.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:38 smithi070 ceph-mon[39923]: osdmap e608: 8 total, 8 up, 8 in 2024-02-10T07:51:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:38 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-120", "mode": "writeback"}]': finished 2024-02-10T07:51:38.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:38 smithi155 ceph-mon[32688]: osdmap e608: 8 total, 8 up, 8 in 2024-02-10T07:51:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:39 smithi070 ceph-mon[29548]: pgmap v927: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:39.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:39.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:39 smithi070 ceph-mon[39923]: pgmap v927: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:39.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:39.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:51:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:51:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:39 smithi155 ceph-mon[32688]: pgmap v927: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:39.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[29548]: osdmap e609: 8 total, 8 up, 8 in 2024-02-10T07:51:40.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120"}]: dispatch 2024-02-10T07:51:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[39923]: osdmap e609: 8 total, 8 up, 8 in 2024-02-10T07:51:40.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120"}]: dispatch 2024-02-10T07:51:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:40 smithi155 ceph-mon[32688]: osdmap e609: 8 total, 8 up, 8 in 2024-02-10T07:51:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120"}]: dispatch 2024-02-10T07:51:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:41 smithi155 ceph-mon[32688]: pgmap v930: 332 pgs: 332 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T07:51:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:41 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120"}]': finished 2024-02-10T07:51:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:41 smithi155 ceph-mon[32688]: osdmap e610: 8 total, 8 up, 8 in 2024-02-10T07:51:41.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[29548]: pgmap v930: 332 pgs: 332 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T07:51:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120"}]': finished 2024-02-10T07:51:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[29548]: osdmap e610: 8 total, 8 up, 8 in 2024-02-10T07:51:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[39923]: pgmap v930: 332 pgs: 332 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T07:51:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-120"}]': finished 2024-02-10T07:51:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[39923]: osdmap e610: 8 total, 8 up, 8 in 2024-02-10T07:51:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:51:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:51:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:51:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:42 smithi155 ceph-mon[32688]: osdmap e611: 8 total, 8 up, 8 in 2024-02-10T07:51:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:42 smithi070 ceph-mon[29548]: osdmap e611: 8 total, 8 up, 8 in 2024-02-10T07:51:42.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:42 smithi070 ceph-mon[39923]: osdmap e611: 8 total, 8 up, 8 in 2024-02-10T07:51:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:43 smithi070 ceph-mon[29548]: pgmap v933: 300 pgs: 300 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:43 smithi070 ceph-mon[29548]: osdmap e612: 8 total, 8 up, 8 in 2024-02-10T07:51:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:43 smithi070 ceph-mon[39923]: pgmap v933: 300 pgs: 300 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:43 smithi070 ceph-mon[39923]: osdmap e612: 8 total, 8 up, 8 in 2024-02-10T07:51:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:43 smithi155 ceph-mon[32688]: pgmap v933: 300 pgs: 300 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:51:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:43 smithi155 ceph-mon[32688]: osdmap e612: 8 total, 8 up, 8 in 2024-02-10T07:51:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:44 smithi070 ceph-mon[39923]: osdmap e613: 8 total, 8 up, 8 in 2024-02-10T07:51:44.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:44 smithi070 ceph-mon[29548]: osdmap e613: 8 total, 8 up, 8 in 2024-02-10T07:51:44.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:44 smithi155 ceph-mon[32688]: osdmap e613: 8 total, 8 up, 8 in 2024-02-10T07:51:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[29548]: pgmap v936: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:51:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[29548]: osdmap e614: 8 total, 8 up, 8 in 2024-02-10T07:51:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-122"}]: dispatch 2024-02-10T07:51:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[39923]: pgmap v936: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:51:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[39923]: osdmap e614: 8 total, 8 up, 8 in 2024-02-10T07:51:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-122"}]: dispatch 2024-02-10T07:51:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:45 smithi155 ceph-mon[32688]: pgmap v936: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:51:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:45 smithi155 ceph-mon[32688]: osdmap e614: 8 total, 8 up, 8 in 2024-02-10T07:51:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-122"}]: dispatch 2024-02-10T07:51:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-122"}]': finished 2024-02-10T07:51:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[29548]: osdmap e615: 8 total, 8 up, 8 in 2024-02-10T07:51:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-122", "mode": "writeback"}]: dispatch 2024-02-10T07:51:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:46.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-122", "mode": "writeback"}]': finished 2024-02-10T07:51:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-122"}]': finished 2024-02-10T07:51:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[39923]: osdmap e615: 8 total, 8 up, 8 in 2024-02-10T07:51:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-122", "mode": "writeback"}]: dispatch 2024-02-10T07:51:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-122", "mode": "writeback"}]': finished 2024-02-10T07:51:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-122"}]': finished 2024-02-10T07:51:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:46 smithi155 ceph-mon[32688]: osdmap e615: 8 total, 8 up, 8 in 2024-02-10T07:51:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-122", "mode": "writeback"}]: dispatch 2024-02-10T07:51:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:46 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-122", "mode": "writeback"}]': finished 2024-02-10T07:51:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:47 smithi070 ceph-mon[29548]: pgmap v939: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:47 smithi070 ceph-mon[29548]: osdmap e616: 8 total, 8 up, 8 in 2024-02-10T07:51:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:47 smithi070 ceph-mon[39923]: pgmap v939: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:47 smithi070 ceph-mon[39923]: osdmap e616: 8 total, 8 up, 8 in 2024-02-10T07:51:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:47 smithi155 ceph-mon[32688]: pgmap v939: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:47 smithi155 ceph-mon[32688]: osdmap e616: 8 total, 8 up, 8 in 2024-02-10T07:51:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:48 smithi155 ceph-mon[32688]: osdmap e617: 8 total, 8 up, 8 in 2024-02-10T07:51:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122"}]: dispatch 2024-02-10T07:51:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:48 smithi070 ceph-mon[29548]: osdmap e617: 8 total, 8 up, 8 in 2024-02-10T07:51:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122"}]: dispatch 2024-02-10T07:51:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:48 smithi070 ceph-mon[39923]: osdmap e617: 8 total, 8 up, 8 in 2024-02-10T07:51:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122"}]: dispatch 2024-02-10T07:51:49.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:51:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:51:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[39923]: pgmap v942: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122"}]': finished 2024-02-10T07:51:50.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[39923]: osdmap e618: 8 total, 8 up, 8 in 2024-02-10T07:51:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[29548]: pgmap v942: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122"}]': finished 2024-02-10T07:51:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:49 smithi070 ceph-mon[29548]: osdmap e618: 8 total, 8 up, 8 in 2024-02-10T07:51:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:49 smithi155 ceph-mon[32688]: pgmap v942: 332 pgs: 332 active+clean; 582 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:51:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:49 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-122"}]': finished 2024-02-10T07:51:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:49 smithi155 ceph-mon[32688]: osdmap e618: 8 total, 8 up, 8 in 2024-02-10T07:51:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[29548]: osdmap e619: 8 total, 8 up, 8 in 2024-02-10T07:51:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[29548]: pgmap v945: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[29548]: osdmap e620: 8 total, 8 up, 8 in 2024-02-10T07:51:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[39923]: osdmap e619: 8 total, 8 up, 8 in 2024-02-10T07:51:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[39923]: pgmap v945: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[39923]: osdmap e620: 8 total, 8 up, 8 in 2024-02-10T07:51:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:51:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:50 smithi155 ceph-mon[32688]: osdmap e619: 8 total, 8 up, 8 in 2024-02-10T07:51:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:50 smithi155 ceph-mon[32688]: pgmap v945: 300 pgs: 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:50 smithi155 ceph-mon[32688]: osdmap e620: 8 total, 8 up, 8 in 2024-02-10T07:51:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:51:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:51:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:51:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:51:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:52 smithi070 ceph-mon[29548]: osdmap e621: 8 total, 8 up, 8 in 2024-02-10T07:51:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:52.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:52 smithi070 ceph-mon[39923]: osdmap e621: 8 total, 8 up, 8 in 2024-02-10T07:51:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:51:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:52 smithi155 ceph-mon[32688]: osdmap e621: 8 total, 8 up, 8 in 2024-02-10T07:51:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:53 smithi070 ceph-mon[29548]: pgmap v948: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:53 smithi070 ceph-mon[29548]: osdmap e622: 8 total, 8 up, 8 in 2024-02-10T07:51:53.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:53 smithi070 ceph-mon[39923]: pgmap v948: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:53 smithi070 ceph-mon[39923]: osdmap e622: 8 total, 8 up, 8 in 2024-02-10T07:51:53.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:53 smithi155 ceph-mon[32688]: pgmap v948: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:51:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:53 smithi155 ceph-mon[32688]: osdmap e622: 8 total, 8 up, 8 in 2024-02-10T07:51:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:51:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[29548]: osdmap e623: 8 total, 8 up, 8 in 2024-02-10T07:51:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-124"}]: dispatch 2024-02-10T07:51:54.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-124"}]': finished 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[29548]: osdmap e624: 8 total, 8 up, 8 in 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-124", "mode": "writeback"}]: dispatch 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[39923]: osdmap e623: 8 total, 8 up, 8 in 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-124"}]: dispatch 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-124"}]': finished 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[39923]: osdmap e624: 8 total, 8 up, 8 in 2024-02-10T07:51:54.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-124", "mode": "writeback"}]: dispatch 2024-02-10T07:51:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:51:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:54 smithi155 ceph-mon[32688]: osdmap e623: 8 total, 8 up, 8 in 2024-02-10T07:51:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-124"}]: dispatch 2024-02-10T07:51:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-124"}]': finished 2024-02-10T07:51:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:54 smithi155 ceph-mon[32688]: osdmap e624: 8 total, 8 up, 8 in 2024-02-10T07:51:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-124", "mode": "writeback"}]: dispatch 2024-02-10T07:51:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[39923]: pgmap v951: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-124", "mode": "writeback"}]': finished 2024-02-10T07:51:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[39923]: osdmap e625: 8 total, 8 up, 8 in 2024-02-10T07:51:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[29548]: pgmap v951: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-124", "mode": "writeback"}]': finished 2024-02-10T07:51:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:55 smithi070 ceph-mon[29548]: osdmap e625: 8 total, 8 up, 8 in 2024-02-10T07:51:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:55 smithi155 ceph-mon[32688]: pgmap v951: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:55 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:51:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-124", "mode": "writeback"}]': finished 2024-02-10T07:51:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:55 smithi155 ceph-mon[32688]: osdmap e625: 8 total, 8 up, 8 in 2024-02-10T07:51:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:56.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:56.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:51:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:51:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:57 smithi070 ceph-mon[29548]: pgmap v954: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:57 smithi070 ceph-mon[29548]: osdmap e626: 8 total, 8 up, 8 in 2024-02-10T07:51:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:57 smithi070 ceph-mon[39923]: pgmap v954: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:57 smithi070 ceph-mon[39923]: osdmap e626: 8 total, 8 up, 8 in 2024-02-10T07:51:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:57 smithi155 ceph-mon[32688]: pgmap v954: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T07:51:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:57 smithi155 ceph-mon[32688]: osdmap e626: 8 total, 8 up, 8 in 2024-02-10T07:51:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:51:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[29548]: osdmap e627: 8 total, 8 up, 8 in 2024-02-10T07:51:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124"}]: dispatch 2024-02-10T07:51:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:58.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124"}]': finished 2024-02-10T07:51:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[39923]: osdmap e627: 8 total, 8 up, 8 in 2024-02-10T07:51:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124"}]: dispatch 2024-02-10T07:51:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:58.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124"}]': finished 2024-02-10T07:51:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:51:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:58 smithi155 ceph-mon[32688]: osdmap e627: 8 total, 8 up, 8 in 2024-02-10T07:51:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124"}]: dispatch 2024-02-10T07:51:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:58 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:51:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-124"}]': finished 2024-02-10T07:51:59.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:51:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:52:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:59 smithi070 ceph-mon[29548]: pgmap v957: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:51:59 smithi070 ceph-mon[29548]: osdmap e628: 8 total, 8 up, 8 in 2024-02-10T07:52:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:59 smithi070 ceph-mon[39923]: pgmap v957: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:00.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:51:59 smithi070 ceph-mon[39923]: osdmap e628: 8 total, 8 up, 8 in 2024-02-10T07:52:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:59 smithi155 ceph-mon[32688]: pgmap v957: 332 pgs: 332 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:51:59 smithi155 ceph-mon[32688]: osdmap e628: 8 total, 8 up, 8 in 2024-02-10T07:52:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[29548]: osdmap e629: 8 total, 8 up, 8 in 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[29548]: pgmap v960: 300 pgs: 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[29548]: osdmap e630: 8 total, 8 up, 8 in 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[39923]: osdmap e629: 8 total, 8 up, 8 in 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[39923]: pgmap v960: 300 pgs: 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[39923]: osdmap e630: 8 total, 8 up, 8 in 2024-02-10T07:52:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:00 smithi155 ceph-mon[32688]: osdmap e629: 8 total, 8 up, 8 in 2024-02-10T07:52:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:00 smithi155 ceph-mon[32688]: pgmap v960: 300 pgs: 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:00 smithi155 ceph-mon[32688]: osdmap e630: 8 total, 8 up, 8 in 2024-02-10T07:52:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:52:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:52:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[39923]: osdmap e631: 8 total, 8 up, 8 in 2024-02-10T07:52:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[39923]: osdmap e632: 8 total, 8 up, 8 in 2024-02-10T07:52:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-126"}]: dispatch 2024-02-10T07:52:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[29548]: osdmap e631: 8 total, 8 up, 8 in 2024-02-10T07:52:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:02.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[29548]: osdmap e632: 8 total, 8 up, 8 in 2024-02-10T07:52:02.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-126"}]: dispatch 2024-02-10T07:52:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:02 smithi155 ceph-mon[32688]: osdmap e631: 8 total, 8 up, 8 in 2024-02-10T07:52:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:02 smithi155 ceph-mon[32688]: osdmap e632: 8 total, 8 up, 8 in 2024-02-10T07:52:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-126"}]: dispatch 2024-02-10T07:52:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[29548]: pgmap v963: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-126"}]': finished 2024-02-10T07:52:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[29548]: osdmap e633: 8 total, 8 up, 8 in 2024-02-10T07:52:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-126", "mode": "writeback"}]: dispatch 2024-02-10T07:52:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[39923]: pgmap v963: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-126"}]': finished 2024-02-10T07:52:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[39923]: osdmap e633: 8 total, 8 up, 8 in 2024-02-10T07:52:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-126", "mode": "writeback"}]: dispatch 2024-02-10T07:52:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:03 smithi155 ceph-mon[32688]: pgmap v963: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-126"}]': finished 2024-02-10T07:52:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:03 smithi155 ceph-mon[32688]: osdmap e633: 8 total, 8 up, 8 in 2024-02-10T07:52:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-126", "mode": "writeback"}]: dispatch 2024-02-10T07:52:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-126", "mode": "writeback"}]': finished 2024-02-10T07:52:04.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[29548]: osdmap e634: 8 total, 8 up, 8 in 2024-02-10T07:52:04.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-126", "mode": "writeback"}]': finished 2024-02-10T07:52:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[39923]: osdmap e634: 8 total, 8 up, 8 in 2024-02-10T07:52:04.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:04 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-126", "mode": "writeback"}]': finished 2024-02-10T07:52:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:04 smithi155 ceph-mon[32688]: osdmap e634: 8 total, 8 up, 8 in 2024-02-10T07:52:05.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[39923]: pgmap v966: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[39923]: osdmap e635: 8 total, 8 up, 8 in 2024-02-10T07:52:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126"}]: dispatch 2024-02-10T07:52:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[29548]: pgmap v966: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[29548]: osdmap e635: 8 total, 8 up, 8 in 2024-02-10T07:52:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126"}]: dispatch 2024-02-10T07:52:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:05 smithi155 ceph-mon[32688]: pgmap v966: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:05 smithi155 ceph-mon[32688]: osdmap e635: 8 total, 8 up, 8 in 2024-02-10T07:52:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126"}]: dispatch 2024-02-10T07:52:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:06 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126"}]': finished 2024-02-10T07:52:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:06 smithi070 ceph-mon[29548]: osdmap e636: 8 total, 8 up, 8 in 2024-02-10T07:52:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:06 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126"}]': finished 2024-02-10T07:52:06.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:06 smithi070 ceph-mon[39923]: osdmap e636: 8 total, 8 up, 8 in 2024-02-10T07:52:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:06 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-126"}]': finished 2024-02-10T07:52:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:06 smithi155 ceph-mon[32688]: osdmap e636: 8 total, 8 up, 8 in 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 se api_tier_pp: require_osd_release = squid 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9060 ms) 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6036 ms) 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5020 ms) 2024-02-10T07:52:07.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25682 ms) 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14050 ms) 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12063 ms) 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: 00000012 2024-02-10T07:52:07.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12077 ms) 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: require_osd_release = squid 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6037 ms) 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (557385 ms total) 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1203 ms) 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10862 ms) 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18070 ms) 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-10T07:52:07.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30137 ms total) 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7036 ms) 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8052 ms) 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: waiting for scrub... 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: done waiting 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24626 ms) 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9902 ms) 2024-02-10T07:52:07.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-10T07:52:07.470 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8044 ms) 2024-02-10T07:52:07.470 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-10T07:52:07.470 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8043 ms) 2024-02-10T07:52:07.470 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-10T07:52:07.470 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10056 ms) 2024-02-10T07:52:07.470 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-10T07:52:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:07 smithi070 ceph-mon[29548]: pgmap v969: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:07 smithi070 ceph-mon[29548]: osdmap e637: 8 total, 8 up, 8 in 2024-02-10T07:52:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:07 smithi070 ceph-mon[39923]: pgmap v969: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:07 smithi070 ceph-mon[39923]: osdmap e637: 8 total, 8 up, 8 in 2024-02-10T07:52:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:07 smithi155 ceph-mon[32688]: pgmap v969: 332 pgs: 332 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:07 smithi155 ceph-mon[32688]: osdmap e637: 8 total, 8 up, 8 in 2024-02-10T07:52:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:09 smithi070 ceph-mon[29548]: pgmap v972: 300 pgs: 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:09 smithi070 ceph-mon[29548]: osdmap e638: 8 total, 8 up, 8 in 2024-02-10T07:52:09.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:09 smithi070 ceph-mon[39923]: pgmap v972: 300 pgs: 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:09 smithi070 ceph-mon[39923]: osdmap e638: 8 total, 8 up, 8 in 2024-02-10T07:52:09.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:09 smithi155 ceph-mon[32688]: pgmap v972: 300 pgs: 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:09 smithi155 ceph-mon[32688]: osdmap e638: 8 total, 8 up, 8 in 2024-02-10T07:52:09.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:09.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:52:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:52:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[29548]: osdmap e639: 8 total, 8 up, 8 in 2024-02-10T07:52:10.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[29548]: osdmap e640: 8 total, 8 up, 8 in 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-128"}]: dispatch 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[39923]: osdmap e639: 8 total, 8 up, 8 in 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:10.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[39923]: osdmap e640: 8 total, 8 up, 8 in 2024-02-10T07:52:10.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-128"}]: dispatch 2024-02-10T07:52:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:10 smithi155 ceph-mon[32688]: osdmap e639: 8 total, 8 up, 8 in 2024-02-10T07:52:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:10 smithi155 ceph-mon[32688]: osdmap e640: 8 total, 8 up, 8 in 2024-02-10T07:52:10.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-128"}]: dispatch 2024-02-10T07:52:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[29548]: pgmap v975: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-128"}]': finished 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[29548]: osdmap e641: 8 total, 8 up, 8 in 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-128", "mode": "writeback"}]: dispatch 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[39923]: pgmap v975: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-128"}]': finished 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[39923]: osdmap e641: 8 total, 8 up, 8 in 2024-02-10T07:52:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-128", "mode": "writeback"}]: dispatch 2024-02-10T07:52:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:11 smithi155 ceph-mon[32688]: pgmap v975: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-128"}]': finished 2024-02-10T07:52:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:11 smithi155 ceph-mon[32688]: osdmap e641: 8 total, 8 up, 8 in 2024-02-10T07:52:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-128", "mode": "writeback"}]: dispatch 2024-02-10T07:52:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:52:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:52:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:52:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[39923]: pgmap v978: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-128", "mode": "writeback"}]': finished 2024-02-10T07:52:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[39923]: osdmap e642: 8 total, 8 up, 8 in 2024-02-10T07:52:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[29548]: pgmap v978: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-128", "mode": "writeback"}]': finished 2024-02-10T07:52:13.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:13 smithi070 ceph-mon[29548]: osdmap e642: 8 total, 8 up, 8 in 2024-02-10T07:52:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:13 smithi155 ceph-mon[32688]: pgmap v978: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:13 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-128", "mode": "writeback"}]': finished 2024-02-10T07:52:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:13 smithi155 ceph-mon[32688]: osdmap e642: 8 total, 8 up, 8 in 2024-02-10T07:52:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:15 smithi070 ceph-mon[29548]: pgmap v980: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 448 B/s wr, 1 op/s 2024-02-10T07:52:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:15 smithi070 ceph-mon[39923]: pgmap v980: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 448 B/s wr, 1 op/s 2024-02-10T07:52:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:15 smithi155 ceph-mon[32688]: pgmap v980: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 448 B/s wr, 1 op/s 2024-02-10T07:52:17.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:17 smithi070 ceph-mon[29548]: pgmap v981: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-10T07:52:17.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:17 smithi070 ceph-mon[39923]: pgmap v981: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-10T07:52:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:17 smithi155 ceph-mon[32688]: pgmap v981: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-10T07:52:18.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:18.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:18.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[29548]: pgmap v982: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 270 B/s wr, 1 op/s 2024-02-10T07:52:19.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[29548]: osdmap e643: 8 total, 8 up, 8 in 2024-02-10T07:52:19.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128"}]: dispatch 2024-02-10T07:52:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[39923]: pgmap v982: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 270 B/s wr, 1 op/s 2024-02-10T07:52:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[39923]: osdmap e643: 8 total, 8 up, 8 in 2024-02-10T07:52:19.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128"}]: dispatch 2024-02-10T07:52:19.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:52:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:52:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:19 smithi155 ceph-mon[32688]: pgmap v982: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 270 B/s wr, 1 op/s 2024-02-10T07:52:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:19 smithi155 ceph-mon[32688]: osdmap e643: 8 total, 8 up, 8 in 2024-02-10T07:52:19.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128"}]: dispatch 2024-02-10T07:52:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128"}]': finished 2024-02-10T07:52:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[29548]: osdmap e644: 8 total, 8 up, 8 in 2024-02-10T07:52:20.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:20.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128"}]': finished 2024-02-10T07:52:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[39923]: osdmap e644: 8 total, 8 up, 8 in 2024-02-10T07:52:20.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:20 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:20 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-128"}]': finished 2024-02-10T07:52:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:20 smithi155 ceph-mon[32688]: osdmap e644: 8 total, 8 up, 8 in 2024-02-10T07:52:20.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:21 smithi070 ceph-mon[29548]: pgmap v985: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 270 B/s wr, 1 op/s 2024-02-10T07:52:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:21 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:21.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:21 smithi070 ceph-mon[29548]: osdmap e645: 8 total, 8 up, 8 in 2024-02-10T07:52:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:21 smithi070 ceph-mon[39923]: pgmap v985: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 270 B/s wr, 1 op/s 2024-02-10T07:52:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:21 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:21.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:21 smithi070 ceph-mon[39923]: osdmap e645: 8 total, 8 up, 8 in 2024-02-10T07:52:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:21 smithi155 ceph-mon[32688]: pgmap v985: 332 pgs: 332 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 270 B/s wr, 1 op/s 2024-02-10T07:52:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:21 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:21.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:21 smithi155 ceph-mon[32688]: osdmap e645: 8 total, 8 up, 8 in 2024-02-10T07:52:22.488 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:52:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:52:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:52:22.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:22 smithi070 ceph-mon[29548]: osdmap e646: 8 total, 8 up, 8 in 2024-02-10T07:52:22.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:22.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:22 smithi070 ceph-mon[39923]: osdmap e646: 8 total, 8 up, 8 in 2024-02-10T07:52:22.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:22 smithi155 ceph-mon[32688]: osdmap e646: 8 total, 8 up, 8 in 2024-02-10T07:52:22.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:23 smithi155 ceph-mon[32688]: pgmap v988: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:52:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:23 smithi155 ceph-mon[32688]: osdmap e647: 8 total, 8 up, 8 in 2024-02-10T07:52:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[29548]: pgmap v988: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:52:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[29548]: osdmap e647: 8 total, 8 up, 8 in 2024-02-10T07:52:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[39923]: pgmap v988: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:52:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[39923]: osdmap e647: 8 total, 8 up, 8 in 2024-02-10T07:52:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:23 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:24 smithi155 ceph-mon[32688]: osdmap e648: 8 total, 8 up, 8 in 2024-02-10T07:52:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-130"}]: dispatch 2024-02-10T07:52:24.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:24 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-130"}]': finished 2024-02-10T07:52:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[29548]: osdmap e648: 8 total, 8 up, 8 in 2024-02-10T07:52:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-130"}]: dispatch 2024-02-10T07:52:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-130"}]': finished 2024-02-10T07:52:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[39923]: osdmap e648: 8 total, 8 up, 8 in 2024-02-10T07:52:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-130"}]: dispatch 2024-02-10T07:52:24.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:24 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-130"}]': finished 2024-02-10T07:52:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:25 smithi155 ceph-mon[32688]: pgmap v991: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:52:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:25 smithi155 ceph-mon[32688]: osdmap e649: 8 total, 8 up, 8 in 2024-02-10T07:52:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-130", "mode": "writeback"}]: dispatch 2024-02-10T07:52:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:25 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[39923]: pgmap v991: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:52:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[39923]: osdmap e649: 8 total, 8 up, 8 in 2024-02-10T07:52:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-130", "mode": "writeback"}]: dispatch 2024-02-10T07:52:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[29548]: pgmap v991: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:52:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[29548]: osdmap e649: 8 total, 8 up, 8 in 2024-02-10T07:52:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-130", "mode": "writeback"}]: dispatch 2024-02-10T07:52:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:25 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-130", "mode": "writeback"}]': finished 2024-02-10T07:52:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:26 smithi155 ceph-mon[32688]: osdmap e650: 8 total, 8 up, 8 in 2024-02-10T07:52:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:26.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-130", "mode": "writeback"}]': finished 2024-02-10T07:52:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[39923]: osdmap e650: 8 total, 8 up, 8 in 2024-02-10T07:52:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-130", "mode": "writeback"}]': finished 2024-02-10T07:52:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[29548]: osdmap e650: 8 total, 8 up, 8 in 2024-02-10T07:52:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[39923]: pgmap v994: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[39923]: osdmap e651: 8 total, 8 up, 8 in 2024-02-10T07:52:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130"}]: dispatch 2024-02-10T07:52:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[29548]: pgmap v994: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[29548]: osdmap e651: 8 total, 8 up, 8 in 2024-02-10T07:52:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130"}]: dispatch 2024-02-10T07:52:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:27 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:27 smithi155 ceph-mon[32688]: pgmap v994: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:27 smithi155 ceph-mon[32688]: osdmap e651: 8 total, 8 up, 8 in 2024-02-10T07:52:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130"}]: dispatch 2024-02-10T07:52:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:27 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:28 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130"}]': finished 2024-02-10T07:52:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:28 smithi070 ceph-mon[39923]: osdmap e652: 8 total, 8 up, 8 in 2024-02-10T07:52:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:28 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130"}]': finished 2024-02-10T07:52:28.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:28 smithi070 ceph-mon[29548]: osdmap e652: 8 total, 8 up, 8 in 2024-02-10T07:52:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:28 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-130"}]': finished 2024-02-10T07:52:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:28 smithi155 ceph-mon[32688]: osdmap e652: 8 total, 8 up, 8 in 2024-02-10T07:52:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:29 smithi070 ceph-mon[29548]: pgmap v997: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:29 smithi070 ceph-mon[29548]: osdmap e653: 8 total, 8 up, 8 in 2024-02-10T07:52:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:29 smithi070 ceph-mon[39923]: pgmap v997: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:29 smithi070 ceph-mon[39923]: osdmap e653: 8 total, 8 up, 8 in 2024-02-10T07:52:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:52:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:52:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:29 smithi155 ceph-mon[32688]: pgmap v997: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:30.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:29 smithi155 ceph-mon[32688]: osdmap e653: 8 total, 8 up, 8 in 2024-02-10T07:52:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:30 smithi070 ceph-mon[29548]: osdmap e654: 8 total, 8 up, 8 in 2024-02-10T07:52:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:30 smithi070 ceph-mon[39923]: osdmap e654: 8 total, 8 up, 8 in 2024-02-10T07:52:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:30 smithi155 ceph-mon[32688]: osdmap e654: 8 total, 8 up, 8 in 2024-02-10T07:52:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[29548]: pgmap v1000: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[29548]: osdmap e655: 8 total, 8 up, 8 in 2024-02-10T07:52:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[39923]: pgmap v1000: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[39923]: osdmap e655: 8 total, 8 up, 8 in 2024-02-10T07:52:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:31 smithi155 ceph-mon[32688]: pgmap v1000: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:31 smithi155 ceph-mon[32688]: osdmap e655: 8 total, 8 up, 8 in 2024-02-10T07:52:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:52:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:52:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[39923]: osdmap e656: 8 total, 8 up, 8 in 2024-02-10T07:52:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:32.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:32.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[29548]: osdmap e656: 8 total, 8 up, 8 in 2024-02-10T07:52:32.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:32.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:32 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:32 smithi155 ceph-mon[32688]: osdmap e656: 8 total, 8 up, 8 in 2024-02-10T07:52:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:33.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:32 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[29548]: pgmap v1003: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[29548]: osdmap e657: 8 total, 8 up, 8 in 2024-02-10T07:52:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-132", "mode": "writeback"}]: dispatch 2024-02-10T07:52:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:52:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[39923]: pgmap v1003: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:33.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[39923]: osdmap e657: 8 total, 8 up, 8 in 2024-02-10T07:52:33.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-132", "mode": "writeback"}]: dispatch 2024-02-10T07:52:33.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:33 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:52:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:33 smithi155 ceph-mon[32688]: pgmap v1003: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:33 smithi155 ceph-mon[32688]: osdmap e657: 8 total, 8 up, 8 in 2024-02-10T07:52:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-132", "mode": "writeback"}]: dispatch 2024-02-10T07:52:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:33 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:52:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-132", "mode": "writeback"}]': finished 2024-02-10T07:52:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[29548]: osdmap e658: 8 total, 8 up, 8 in 2024-02-10T07:52:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:52:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:52:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:34.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:34.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-132", "mode": "writeback"}]': finished 2024-02-10T07:52:34.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[39923]: osdmap e658: 8 total, 8 up, 8 in 2024-02-10T07:52:34.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:52:34.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:52:34.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:34 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:34 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-132", "mode": "writeback"}]': finished 2024-02-10T07:52:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:34 smithi155 ceph-mon[32688]: osdmap e658: 8 total, 8 up, 8 in 2024-02-10T07:52:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:34 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:52:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:34 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:52:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:34 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:35 smithi070 ceph-mon[29548]: pgmap v1006: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:35 smithi070 ceph-mon[29548]: osdmap e659: 8 total, 8 up, 8 in 2024-02-10T07:52:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:35 smithi070 ceph-mon[39923]: pgmap v1006: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:35 smithi070 ceph-mon[39923]: osdmap e659: 8 total, 8 up, 8 in 2024-02-10T07:52:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:35 smithi155 ceph-mon[32688]: pgmap v1006: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:35 smithi155 ceph-mon[32688]: osdmap e659: 8 total, 8 up, 8 in 2024-02-10T07:52:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:36 smithi070 ceph-mon[29548]: osdmap e660: 8 total, 8 up, 8 in 2024-02-10T07:52:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:36 smithi070 ceph-mon[39923]: osdmap e660: 8 total, 8 up, 8 in 2024-02-10T07:52:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:36 smithi155 ceph-mon[32688]: osdmap e660: 8 total, 8 up, 8 in 2024-02-10T07:52:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:37 smithi155 ceph-mon[32688]: pgmap v1009: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:37 smithi155 ceph-mon[32688]: osdmap e661: 8 total, 8 up, 8 in 2024-02-10T07:52:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:37 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[29548]: pgmap v1009: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[29548]: osdmap e661: 8 total, 8 up, 8 in 2024-02-10T07:52:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[39923]: pgmap v1009: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T07:52:37.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:37.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[39923]: osdmap e661: 8 total, 8 up, 8 in 2024-02-10T07:52:37.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:37.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:37 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:52:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:38 smithi070 ceph-mon[29548]: osdmap e662: 8 total, 8 up, 8 in 2024-02-10T07:52:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:38 smithi070 ceph-mon[39923]: osdmap e662: 8 total, 8 up, 8 in 2024-02-10T07:52:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:38 smithi155 ceph-mon[32688]: osdmap e662: 8 total, 8 up, 8 in 2024-02-10T07:52:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:39 smithi070 ceph-mon[39923]: pgmap v1012: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:39 smithi070 ceph-mon[39923]: osdmap e663: 8 total, 8 up, 8 in 2024-02-10T07:52:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:39 smithi070 ceph-mon[29548]: pgmap v1012: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:39 smithi070 ceph-mon[29548]: osdmap e663: 8 total, 8 up, 8 in 2024-02-10T07:52:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:40.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:52:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:52:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:39 smithi155 ceph-mon[32688]: pgmap v1012: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:39 smithi155 ceph-mon[32688]: osdmap e663: 8 total, 8 up, 8 in 2024-02-10T07:52:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[29548]: osdmap e664: 8 total, 8 up, 8 in 2024-02-10T07:52:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[39923]: osdmap e664: 8 total, 8 up, 8 in 2024-02-10T07:52:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:40 smithi155 ceph-mon[32688]: osdmap e664: 8 total, 8 up, 8 in 2024-02-10T07:52:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132"}]: dispatch 2024-02-10T07:52:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[39923]: pgmap v1015: 332 pgs: 332 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:52:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[39923]: osdmap e665: 8 total, 8 up, 8 in 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[29548]: pgmap v1015: 332 pgs: 332 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[29548]: osdmap e665: 8 total, 8 up, 8 in 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:41 smithi155 ceph-mon[32688]: pgmap v1015: 332 pgs: 332 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T07:52:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:41 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-132"}]': finished 2024-02-10T07:52:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:41 smithi155 ceph-mon[32688]: osdmap e665: 8 total, 8 up, 8 in 2024-02-10T07:52:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:52:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:52:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:52:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:42 smithi070 ceph-mon[29548]: osdmap e666: 8 total, 8 up, 8 in 2024-02-10T07:52:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:42 smithi070 ceph-mon[29548]: osdmap e667: 8 total, 8 up, 8 in 2024-02-10T07:52:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:42 smithi070 ceph-mon[39923]: osdmap e666: 8 total, 8 up, 8 in 2024-02-10T07:52:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:42 smithi070 ceph-mon[39923]: osdmap e667: 8 total, 8 up, 8 in 2024-02-10T07:52:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:42 smithi155 ceph-mon[32688]: osdmap e666: 8 total, 8 up, 8 in 2024-02-10T07:52:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:42 smithi155 ceph-mon[32688]: osdmap e667: 8 total, 8 up, 8 in 2024-02-10T07:52:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[29548]: pgmap v1018: 300 pgs: 300 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:52:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[29548]: osdmap e668: 8 total, 8 up, 8 in 2024-02-10T07:52:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[39923]: pgmap v1018: 300 pgs: 300 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:52:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[39923]: osdmap e668: 8 total, 8 up, 8 in 2024-02-10T07:52:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:43 smithi155 ceph-mon[32688]: pgmap v1018: 300 pgs: 300 active+clean; 582 KiB data, 704 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T07:52:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:43 smithi155 ceph-mon[32688]: osdmap e668: 8 total, 8 up, 8 in 2024-02-10T07:52:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[29548]: pgmap v1021: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:52:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[29548]: osdmap e669: 8 total, 8 up, 8 in 2024-02-10T07:52:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-134"}]: dispatch 2024-02-10T07:52:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[39923]: pgmap v1021: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:52:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[39923]: osdmap e669: 8 total, 8 up, 8 in 2024-02-10T07:52:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-134"}]: dispatch 2024-02-10T07:52:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:45 smithi155 ceph-mon[32688]: pgmap v1021: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T07:52:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:45 smithi155 ceph-mon[32688]: osdmap e669: 8 total, 8 up, 8 in 2024-02-10T07:52:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-134"}]: dispatch 2024-02-10T07:52:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-134"}]': finished 2024-02-10T07:52:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[29548]: osdmap e670: 8 total, 8 up, 8 in 2024-02-10T07:52:46.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-134", "mode": "writeback"}]: dispatch 2024-02-10T07:52:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:46.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-134", "mode": "writeback"}]': finished 2024-02-10T07:52:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-134"}]': finished 2024-02-10T07:52:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[39923]: osdmap e670: 8 total, 8 up, 8 in 2024-02-10T07:52:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-134", "mode": "writeback"}]: dispatch 2024-02-10T07:52:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:46.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-134", "mode": "writeback"}]': finished 2024-02-10T07:52:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-134"}]': finished 2024-02-10T07:52:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:46 smithi155 ceph-mon[32688]: osdmap e670: 8 total, 8 up, 8 in 2024-02-10T07:52:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-134", "mode": "writeback"}]: dispatch 2024-02-10T07:52:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:46 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:46.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-134", "mode": "writeback"}]': finished 2024-02-10T07:52:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:47 smithi070 ceph-mon[29548]: pgmap v1024: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:47 smithi070 ceph-mon[29548]: osdmap e671: 8 total, 8 up, 8 in 2024-02-10T07:52:47.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:47 smithi070 ceph-mon[39923]: pgmap v1024: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:47 smithi070 ceph-mon[39923]: osdmap e671: 8 total, 8 up, 8 in 2024-02-10T07:52:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:47 smithi155 ceph-mon[32688]: pgmap v1024: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:47 smithi155 ceph-mon[32688]: osdmap e671: 8 total, 8 up, 8 in 2024-02-10T07:52:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:48 smithi070 ceph-mon[29548]: osdmap e672: 8 total, 8 up, 8 in 2024-02-10T07:52:48.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134"}]: dispatch 2024-02-10T07:52:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:48 smithi070 ceph-mon[39923]: osdmap e672: 8 total, 8 up, 8 in 2024-02-10T07:52:48.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134"}]: dispatch 2024-02-10T07:52:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:48 smithi155 ceph-mon[32688]: osdmap e672: 8 total, 8 up, 8 in 2024-02-10T07:52:48.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134"}]: dispatch 2024-02-10T07:52:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[39923]: pgmap v1027: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134"}]': finished 2024-02-10T07:52:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[39923]: osdmap e673: 8 total, 8 up, 8 in 2024-02-10T07:52:49.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[29548]: pgmap v1027: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:49.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:49.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134"}]': finished 2024-02-10T07:52:49.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:49 smithi070 ceph-mon[29548]: osdmap e673: 8 total, 8 up, 8 in 2024-02-10T07:52:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:52:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:52:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:49 smithi155 ceph-mon[32688]: pgmap v1027: 332 pgs: 332 active+clean; 582 KiB data, 705 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:49 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:49 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-134"}]': finished 2024-02-10T07:52:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:49 smithi155 ceph-mon[32688]: osdmap e673: 8 total, 8 up, 8 in 2024-02-10T07:52:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[29548]: osdmap e674: 8 total, 8 up, 8 in 2024-02-10T07:52:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[29548]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 706 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:52:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[29548]: osdmap e675: 8 total, 8 up, 8 in 2024-02-10T07:52:50.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:50.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[39923]: osdmap e674: 8 total, 8 up, 8 in 2024-02-10T07:52:50.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:50.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[39923]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 706 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:52:50.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[39923]: osdmap e675: 8 total, 8 up, 8 in 2024-02-10T07:52:50.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:50 smithi155 ceph-mon[32688]: osdmap e674: 8 total, 8 up, 8 in 2024-02-10T07:52:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:52:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:50 smithi155 ceph-mon[32688]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 706 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:52:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:50 smithi155 ceph-mon[32688]: osdmap e675: 8 total, 8 up, 8 in 2024-02-10T07:52:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:52:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:52:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:52:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:52 smithi155 ceph-mon[32688]: osdmap e676: 8 total, 8 up, 8 in 2024-02-10T07:52:52.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:52.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:52.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:52 smithi070 ceph-mon[29548]: osdmap e676: 8 total, 8 up, 8 in 2024-02-10T07:52:52.757 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:52.757 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:52:52.757 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:52 smithi070 ceph-mon[39923]: osdmap e676: 8 total, 8 up, 8 in 2024-02-10T07:52:52.758 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:52:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:53 smithi155 ceph-mon[32688]: pgmap v1033: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 706 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:52:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:53 smithi155 ceph-mon[32688]: osdmap e677: 8 total, 8 up, 8 in 2024-02-10T07:52:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-136"}]: dispatch 2024-02-10T07:52:53.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-136"}]': finished 2024-02-10T07:52:53.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:53 smithi155 ceph-mon[32688]: osdmap e678: 8 total, 8 up, 8 in 2024-02-10T07:52:53.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-136", "mode": "writeback"}]: dispatch 2024-02-10T07:52:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[39923]: pgmap v1033: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 706 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:52:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[39923]: osdmap e677: 8 total, 8 up, 8 in 2024-02-10T07:52:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-136"}]: dispatch 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-136"}]': finished 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[39923]: osdmap e678: 8 total, 8 up, 8 in 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-136", "mode": "writeback"}]: dispatch 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[29548]: pgmap v1033: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 706 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[29548]: osdmap e677: 8 total, 8 up, 8 in 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-136"}]: dispatch 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-136"}]': finished 2024-02-10T07:52:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[29548]: osdmap e678: 8 total, 8 up, 8 in 2024-02-10T07:52:53.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-136", "mode": "writeback"}]: dispatch 2024-02-10T07:52:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:54 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:54.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-136", "mode": "writeback"}]': finished 2024-02-10T07:52:54.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:54 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:54.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-136", "mode": "writeback"}]': finished 2024-02-10T07:52:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:54 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:52:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-136", "mode": "writeback"}]': finished 2024-02-10T07:52:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:55 smithi070 ceph-mon[29548]: pgmap v1036: 332 pgs: 332 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:55 smithi070 ceph-mon[29548]: osdmap e679: 8 total, 8 up, 8 in 2024-02-10T07:52:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:55 smithi070 ceph-mon[39923]: pgmap v1036: 332 pgs: 332 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:55 smithi070 ceph-mon[39923]: osdmap e679: 8 total, 8 up, 8 in 2024-02-10T07:52:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:55 smithi155 ceph-mon[32688]: pgmap v1036: 332 pgs: 332 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:55 smithi155 ceph-mon[32688]: osdmap e679: 8 total, 8 up, 8 in 2024-02-10T07:52:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:52:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[29548]: osdmap e680: 8 total, 8 up, 8 in 2024-02-10T07:52:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136"}]: dispatch 2024-02-10T07:52:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[39923]: osdmap e680: 8 total, 8 up, 8 in 2024-02-10T07:52:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136"}]: dispatch 2024-02-10T07:52:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:52:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:56 smithi155 ceph-mon[32688]: osdmap e680: 8 total, 8 up, 8 in 2024-02-10T07:52:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136"}]: dispatch 2024-02-10T07:52:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:52:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:52:57.461 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ]c 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.462 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.463 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.464 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.465 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.466 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.467 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.468 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.469 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:52:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[29548]: pgmap v1039: 332 pgs: 332 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136"}]': finished 2024-02-10T07:52:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[29548]: osdmap e681: 8 total, 8 up, 8 in 2024-02-10T07:52:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[39923]: pgmap v1039: 332 pgs: 332 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136"}]': finished 2024-02-10T07:52:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:57 smithi070 ceph-mon[39923]: osdmap e681: 8 total, 8 up, 8 in 2024-02-10T07:52:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:57 smithi155 ceph-mon[32688]: pgmap v1039: 332 pgs: 332 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:52:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:57 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:52:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-136"}]': finished 2024-02-10T07:52:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:57 smithi155 ceph-mon[32688]: osdmap e681: 8 total, 8 up, 8 in 2024-02-10T07:52:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:58 smithi070 ceph-mon[29548]: osdmap e682: 8 total, 8 up, 8 in 2024-02-10T07:52:58.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:58 smithi070 ceph-mon[39923]: osdmap e682: 8 total, 8 up, 8 in 2024-02-10T07:52:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:58 smithi155 ceph-mon[32688]: osdmap e682: 8 total, 8 up, 8 in 2024-02-10T07:52:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:59 smithi070 ceph-mon[39923]: pgmap v1042: 300 pgs: 300 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:59 smithi070 ceph-mon[39923]: osdmap e683: 8 total, 8 up, 8 in 2024-02-10T07:52:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:52:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:52:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:59 smithi070 ceph-mon[29548]: pgmap v1042: 300 pgs: 300 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:52:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:59 smithi070 ceph-mon[29548]: osdmap e683: 8 total, 8 up, 8 in 2024-02-10T07:52:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:52:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:00.006 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:52:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:53:00.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:59 smithi155 ceph-mon[32688]: pgmap v1042: 300 pgs: 300 active+clean; 582 KiB data, 724 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:53:00.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:59 smithi155 ceph-mon[32688]: osdmap e683: 8 total, 8 up, 8 in 2024-02-10T07:53:00.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:52:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[39923]: osdmap e684: 8 total, 8 up, 8 in 2024-02-10T07:53:00.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[29548]: osdmap e684: 8 total, 8 up, 8 in 2024-02-10T07:53:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:00 smithi155 ceph-mon[32688]: osdmap e684: 8 total, 8 up, 8 in 2024-02-10T07:53:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[29548]: pgmap v1045: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:53:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[29548]: osdmap e685: 8 total, 8 up, 8 in 2024-02-10T07:53:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:53:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[39923]: pgmap v1045: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:53:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[39923]: osdmap e685: 8 total, 8 up, 8 in 2024-02-10T07:53:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:53:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:01 smithi155 ceph-mon[32688]: pgmap v1045: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:53:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:01 smithi155 ceph-mon[32688]: osdmap e685: 8 total, 8 up, 8 in 2024-02-10T07:53:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:53:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:53:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:53:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:53:02.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[39923]: osdmap e686: 8 total, 8 up, 8 in 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[39923]: osdmap e687: 8 total, 8 up, 8 in 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[29548]: osdmap e686: 8 total, 8 up, 8 in 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[29548]: osdmap e687: 8 total, 8 up, 8 in 2024-02-10T07:53:02.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:53:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:53:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:02 smithi155 ceph-mon[32688]: osdmap e686: 8 total, 8 up, 8 in 2024-02-10T07:53:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:53:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:53:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:02 smithi155 ceph-mon[32688]: osdmap e687: 8 total, 8 up, 8 in 2024-02-10T07:53:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T07:53:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:03 smithi070 ceph-mon[39923]: pgmap v1048: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:53:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:53:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:03 smithi070 ceph-mon[39923]: osdmap e688: 8 total, 8 up, 8 in 2024-02-10T07:53:03.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:03 smithi070 ceph-mon[29548]: pgmap v1048: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:53:03.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:53:03.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:03 smithi070 ceph-mon[29548]: osdmap e688: 8 total, 8 up, 8 in 2024-02-10T07:53:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:03 smithi155 ceph-mon[32688]: pgmap v1048: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-10T07:53:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T07:53:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:03 smithi155 ceph-mon[32688]: osdmap e688: 8 total, 8 up, 8 in 2024-02-10T07:53:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:05 smithi070 ceph-mon[29548]: pgmap v1051: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138"}]: dispatch 2024-02-10T07:53:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:05 smithi070 ceph-mon[39923]: pgmap v1051: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138"}]: dispatch 2024-02-10T07:53:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:05 smithi155 ceph-mon[32688]: pgmap v1051: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138"}]: dispatch 2024-02-10T07:53:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138"}]': finished 2024-02-10T07:53:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:06 smithi070 ceph-mon[29548]: osdmap e689: 8 total, 8 up, 8 in 2024-02-10T07:53:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138"}]': finished 2024-02-10T07:53:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:06 smithi070 ceph-mon[39923]: osdmap e689: 8 total, 8 up, 8 in 2024-02-10T07:53:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-138"}]': finished 2024-02-10T07:53:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:06 smithi155 ceph-mon[32688]: osdmap e689: 8 total, 8 up, 8 in 2024-02-10T07:53:07.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:07 smithi155 ceph-mon[32688]: pgmap v1053: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:53:07.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:07 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:07.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:07 smithi155 ceph-mon[32688]: osdmap e690: 8 total, 8 up, 8 in 2024-02-10T07:53:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:07 smithi070 ceph-mon[39923]: pgmap v1053: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:53:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:07 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:07 smithi070 ceph-mon[39923]: osdmap e690: 8 total, 8 up, 8 in 2024-02-10T07:53:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:07 smithi070 ceph-mon[29548]: pgmap v1053: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:53:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:07 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:07 smithi070 ceph-mon[29548]: osdmap e690: 8 total, 8 up, 8 in 2024-02-10T07:53:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[39923]: osdmap e691: 8 total, 8 up, 8 in 2024-02-10T07:53:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[39923]: osdmap e692: 8 total, 8 up, 8 in 2024-02-10T07:53:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[29548]: osdmap e691: 8 total, 8 up, 8 in 2024-02-10T07:53:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:08 smithi070 ceph-mon[29548]: osdmap e692: 8 total, 8 up, 8 in 2024-02-10T07:53:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:08 smithi155 ceph-mon[32688]: osdmap e691: 8 total, 8 up, 8 in 2024-02-10T07:53:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:08 smithi155 ceph-mon[32688]: osdmap e692: 8 total, 8 up, 8 in 2024-02-10T07:53:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[39923]: pgmap v1056: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:53:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[39923]: osdmap e693: 8 total, 8 up, 8 in 2024-02-10T07:53:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:53:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[29548]: pgmap v1056: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:53:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[29548]: osdmap e693: 8 total, 8 up, 8 in 2024-02-10T07:53:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:53:10.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:53:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:53:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:09 smithi155 ceph-mon[32688]: pgmap v1056: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T07:53:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:09 smithi155 ceph-mon[32688]: osdmap e693: 8 total, 8 up, 8 in 2024-02-10T07:53:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T07:53:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_count","val": "3"}]': finished 2024-02-10T07:53:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[29548]: osdmap e694: 8 total, 8 up, 8 in 2024-02-10T07:53:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:53:10.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:10.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_count","val": "3"}]': finished 2024-02-10T07:53:10.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[39923]: osdmap e694: 8 total, 8 up, 8 in 2024-02-10T07:53:10.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:53:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_count","val": "3"}]': finished 2024-02-10T07:53:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:10 smithi155 ceph-mon[32688]: osdmap e694: 8 total, 8 up, 8 in 2024-02-10T07:53:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T07:53:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[29548]: pgmap v1059: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_period","val": "3"}]': finished 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[29548]: osdmap e695: 8 total, 8 up, 8 in 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[39923]: pgmap v1059: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_period","val": "3"}]': finished 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[39923]: osdmap e695: 8 total, 8 up, 8 in 2024-02-10T07:53:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:11 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:53:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:11 smithi155 ceph-mon[32688]: pgmap v1059: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_period","val": "3"}]': finished 2024-02-10T07:53:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:11 smithi155 ceph-mon[32688]: osdmap e695: 8 total, 8 up, 8 in 2024-02-10T07:53:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:11 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:53:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:53:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:53:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:53:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[29548]: osdmap e696: 8 total, 8 up, 8 in 2024-02-10T07:53:12.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:53:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:53:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[39923]: osdmap e696: 8 total, 8 up, 8 in 2024-02-10T07:53:12.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:12 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:53:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:12 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:53:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:12 smithi155 ceph-mon[32688]: osdmap e696: 8 total, 8 up, 8 in 2024-02-10T07:53:13.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:12 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T07:53:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:13 smithi070 ceph-mon[29548]: pgmap v1062: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:13 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T07:53:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:13 smithi070 ceph-mon[29548]: osdmap e697: 8 total, 8 up, 8 in 2024-02-10T07:53:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:13 smithi070 ceph-mon[39923]: pgmap v1062: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:13 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T07:53:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:13 smithi070 ceph-mon[39923]: osdmap e697: 8 total, 8 up, 8 in 2024-02-10T07:53:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:13 smithi155 ceph-mon[32688]: pgmap v1062: 332 pgs: 332 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:13 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T07:53:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:13 smithi155 ceph-mon[32688]: osdmap e697: 8 total, 8 up, 8 in 2024-02-10T07:53:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:15 smithi070 ceph-mon[29548]: pgmap v1065: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:53:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:15 smithi070 ceph-mon[39923]: pgmap v1065: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:53:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:15 smithi155 ceph-mon[32688]: pgmap v1065: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-10T07:53:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:17 smithi070 ceph-mon[29548]: pgmap v1066: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 936 B/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:53:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:17 smithi070 ceph-mon[39923]: pgmap v1066: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 936 B/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:53:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:17 smithi155 ceph-mon[32688]: pgmap v1066: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 936 B/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T07:53:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:19 smithi070 ceph-mon[29548]: pgmap v1067: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-02-10T07:53:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:19 smithi070 ceph-mon[39923]: pgmap v1067: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-02-10T07:53:20.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:53:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:53:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:19 smithi155 ceph-mon[32688]: pgmap v1067: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 792 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-02-10T07:53:20.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:21 smithi070 ceph-mon[29548]: pgmap v1068: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-02-10T07:53:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:21 smithi070 ceph-mon[39923]: pgmap v1068: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-02-10T07:53:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:21 smithi155 ceph-mon[32688]: pgmap v1068: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-02-10T07:53:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:53:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:53:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:53:23.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:23 smithi070 ceph-mon[29548]: pgmap v1069: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.2 KiB/s wr, 1 op/s 2024-02-10T07:53:23.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:23 smithi070 ceph-mon[39923]: pgmap v1069: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.2 KiB/s wr, 1 op/s 2024-02-10T07:53:23.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:23 smithi155 ceph-mon[32688]: pgmap v1069: 332 pgs: 332 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.2 KiB/s wr, 1 op/s 2024-02-10T07:53:25.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:25 smithi070 ceph-mon[39923]: pgmap v1070: 332 pgs: 332 active+clean; 4.5 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 8.7 KiB/s wr, 2 op/s 2024-02-10T07:53:25.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:25 smithi070 ceph-mon[29548]: pgmap v1070: 332 pgs: 332 active+clean; 4.5 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 8.7 KiB/s wr, 2 op/s 2024-02-10T07:53:25.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:25 smithi155 ceph-mon[32688]: pgmap v1070: 332 pgs: 332 active+clean; 4.5 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 8.7 KiB/s wr, 2 op/s 2024-02-10T07:53:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140"}]: dispatch 2024-02-10T07:53:26.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140"}]: dispatch 2024-02-10T07:53:26.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140"}]: dispatch 2024-02-10T07:53:26.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:27 smithi070 ceph-mon[39923]: pgmap v1071: 332 pgs: 332 active+clean; 4.5 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-02-10T07:53:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140"}]': finished 2024-02-10T07:53:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:27 smithi070 ceph-mon[39923]: osdmap e698: 8 total, 8 up, 8 in 2024-02-10T07:53:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:27 smithi070 ceph-mon[29548]: pgmap v1071: 332 pgs: 332 active+clean; 4.5 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-02-10T07:53:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140"}]': finished 2024-02-10T07:53:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:27 smithi070 ceph-mon[29548]: osdmap e698: 8 total, 8 up, 8 in 2024-02-10T07:53:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:27 smithi155 ceph-mon[32688]: pgmap v1071: 332 pgs: 332 active+clean; 4.5 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-02-10T07:53:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-140"}]': finished 2024-02-10T07:53:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:27 smithi155 ceph-mon[32688]: osdmap e698: 8 total, 8 up, 8 in 2024-02-10T07:53:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[29548]: osdmap e699: 8 total, 8 up, 8 in 2024-02-10T07:53:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[29548]: pgmap v1074: 300 pgs: 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:53:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[29548]: osdmap e700: 8 total, 8 up, 8 in 2024-02-10T07:53:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[39923]: osdmap e699: 8 total, 8 up, 8 in 2024-02-10T07:53:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[39923]: pgmap v1074: 300 pgs: 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:53:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[39923]: osdmap e700: 8 total, 8 up, 8 in 2024-02-10T07:53:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:29.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:28 smithi155 ceph-mon[32688]: osdmap e699: 8 total, 8 up, 8 in 2024-02-10T07:53:29.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:28 smithi155 ceph-mon[32688]: pgmap v1074: 300 pgs: 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T07:53:29.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:28 smithi155 ceph-mon[32688]: osdmap e700: 8 total, 8 up, 8 in 2024-02-10T07:53:29.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:53:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:53:30.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:30.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:30 smithi155 ceph-mon[32688]: osdmap e701: 8 total, 8 up, 8 in 2024-02-10T07:53:30.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:30.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[29548]: osdmap e701: 8 total, 8 up, 8 in 2024-02-10T07:53:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:30.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[39923]: osdmap e701: 8 total, 8 up, 8 in 2024-02-10T07:53:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[29548]: pgmap v1077: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[29548]: osdmap e702: 8 total, 8 up, 8 in 2024-02-10T07:53:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[39923]: pgmap v1077: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[39923]: osdmap e702: 8 total, 8 up, 8 in 2024-02-10T07:53:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:31 smithi155 ceph-mon[32688]: pgmap v1077: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:31 smithi155 ceph-mon[32688]: osdmap e702: 8 total, 8 up, 8 in 2024-02-10T07:53:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:53:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:53:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-142"}]': finished 2024-02-10T07:53:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[39923]: osdmap e703: 8 total, 8 up, 8 in 2024-02-10T07:53:32.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-142", "mode": "writeback"}]: dispatch 2024-02-10T07:53:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:53:32.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-142", "mode": "writeback"}]': finished 2024-02-10T07:53:32.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-142"}]': finished 2024-02-10T07:53:32.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[29548]: osdmap e703: 8 total, 8 up, 8 in 2024-02-10T07:53:32.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-142", "mode": "writeback"}]: dispatch 2024-02-10T07:53:32.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:53:32.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-142", "mode": "writeback"}]': finished 2024-02-10T07:53:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-142"}]': finished 2024-02-10T07:53:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:32 smithi155 ceph-mon[32688]: osdmap e703: 8 total, 8 up, 8 in 2024-02-10T07:53:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-142", "mode": "writeback"}]: dispatch 2024-02-10T07:53:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:32 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:53:33.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-142", "mode": "writeback"}]': finished 2024-02-10T07:53:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[29548]: pgmap v1080: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[29548]: osdmap e704: 8 total, 8 up, 8 in 2024-02-10T07:53:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:53:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:53:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[29548]: osdmap e705: 8 total, 8 up, 8 in 2024-02-10T07:53:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:53:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[39923]: pgmap v1080: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[39923]: osdmap e704: 8 total, 8 up, 8 in 2024-02-10T07:53:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:53:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:53:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[39923]: osdmap e705: 8 total, 8 up, 8 in 2024-02-10T07:53:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:53:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:33 smithi155 ceph-mon[32688]: pgmap v1080: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 732 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:53:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:33 smithi155 ceph-mon[32688]: osdmap e704: 8 total, 8 up, 8 in 2024-02-10T07:53:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:53:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:53:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:33 smithi155 ceph-mon[32688]: osdmap e705: 8 total, 8 up, 8 in 2024-02-10T07:53:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:53:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:53:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[29548]: osdmap e706: 8 total, 8 up, 8 in 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[39923]: osdmap e706: 8 total, 8 up, 8 in 2024-02-10T07:53:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:53:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:34 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:53:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:34 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:34 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:53:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:34 smithi155 ceph-mon[32688]: osdmap e706: 8 total, 8 up, 8 in 2024-02-10T07:53:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:53:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: pgmap v1083: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:53:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: osdmap e707: 8 total, 8 up, 8 in 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: pgmap v1083: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:53:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:53:35.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:35.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:53:35.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:53:35.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: osdmap e707: 8 total, 8 up, 8 in 2024-02-10T07:53:35.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:53:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: pgmap v1083: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:53:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:53:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:53:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:36.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:53:36.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:53:36.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: osdmap e707: 8 total, 8 up, 8 in 2024-02-10T07:53:36.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:53:37.807 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:37 smithi155 ceph-mon[32688]: pgmap v1086: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:53:37.808 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:53:37.808 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:37 smithi155 ceph-mon[32688]: osdmap e708: 8 total, 8 up, 8 in 2024-02-10T07:53:37.808 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:53:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[29548]: pgmap v1086: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:53:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:53:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[29548]: osdmap e708: 8 total, 8 up, 8 in 2024-02-10T07:53:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:53:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[39923]: pgmap v1086: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T07:53:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:53:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[39923]: osdmap e708: 8 total, 8 up, 8 in 2024-02-10T07:53:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T07:53:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T07:53:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: osdmap e709: 8 total, 8 up, 8 in 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.1", "id": [4, 7]}]: dispatch 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.17", "id": [0, 2]}]: dispatch 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.19", "id": [3, 5]}]: dispatch 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.1", "id": [4, 7]}]': finished 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.17", "id": [0, 2]}]': finished 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.19", "id": [3, 5]}]': finished 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[29548]: osdmap e710: 8 total, 8 up, 8 in 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: osdmap e709: 8 total, 8 up, 8 in 2024-02-10T07:53:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.1", "id": [4, 7]}]: dispatch 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.17", "id": [0, 2]}]: dispatch 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.19", "id": [3, 5]}]: dispatch 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.1", "id": [4, 7]}]': finished 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.17", "id": [0, 2]}]': finished 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.19", "id": [3, 5]}]': finished 2024-02-10T07:53:38.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:38 smithi070 ceph-mon[39923]: osdmap e710: 8 total, 8 up, 8 in 2024-02-10T07:53:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T07:53:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: osdmap e709: 8 total, 8 up, 8 in 2024-02-10T07:53:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.1", "id": [4, 7]}]: dispatch 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.17", "id": [0, 2]}]: dispatch 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.19", "id": [3, 5]}]: dispatch 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.1", "id": [4, 7]}]': finished 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.17", "id": [0, 2]}]': finished 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "327.19", "id": [3, 5]}]': finished 2024-02-10T07:53:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:38 smithi155 ceph-mon[32688]: osdmap e710: 8 total, 8 up, 8 in 2024-02-10T07:53:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:39 smithi070 ceph-mon[29548]: pgmap v1089: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:53:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:39 smithi070 ceph-mon[39923]: pgmap v1089: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:53:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:39 smithi155 ceph-mon[32688]: pgmap v1089: 332 pgs: 332 active+clean; 4.4 MiB data, 733 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:53:40.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:53:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:53:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:40 smithi070 ceph-mon[29548]: osdmap e711: 8 total, 8 up, 8 in 2024-02-10T07:53:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:40 smithi070 ceph-mon[39923]: osdmap e711: 8 total, 8 up, 8 in 2024-02-10T07:53:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:40 smithi155 ceph-mon[32688]: osdmap e711: 8 total, 8 up, 8 in 2024-02-10T07:53:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[29548]: pgmap v1092: 332 pgs: 3 peering, 329 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:53:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[29548]: osdmap e712: 8 total, 8 up, 8 in 2024-02-10T07:53:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[39923]: pgmap v1092: 332 pgs: 3 peering, 329 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:53:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[39923]: osdmap e712: 8 total, 8 up, 8 in 2024-02-10T07:53:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:41 smithi155 ceph-mon[32688]: pgmap v1092: 332 pgs: 3 peering, 329 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:53:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:41 smithi155 ceph-mon[32688]: osdmap e712: 8 total, 8 up, 8 in 2024-02-10T07:53:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:53:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:53:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]': finished 2024-02-10T07:53:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[29548]: osdmap e713: 8 total, 8 up, 8 in 2024-02-10T07:53:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:42.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]': finished 2024-02-10T07:53:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[39923]: osdmap e713: 8 total, 8 up, 8 in 2024-02-10T07:53:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]': finished 2024-02-10T07:53:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:42 smithi155 ceph-mon[32688]: osdmap e713: 8 total, 8 up, 8 in 2024-02-10T07:53:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-142"}]: dispatch 2024-02-10T07:53:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:43 smithi070 ceph-mon[29548]: pgmap v1095: 332 pgs: 3 peering, 329 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:43 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:43 smithi070 ceph-mon[29548]: osdmap e714: 8 total, 8 up, 8 in 2024-02-10T07:53:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:43 smithi070 ceph-mon[39923]: pgmap v1095: 332 pgs: 3 peering, 329 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:43 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:43 smithi070 ceph-mon[39923]: osdmap e714: 8 total, 8 up, 8 in 2024-02-10T07:53:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:43 smithi155 ceph-mon[32688]: pgmap v1095: 332 pgs: 3 peering, 329 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:43 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:43 smithi155 ceph-mon[32688]: osdmap e714: 8 total, 8 up, 8 in 2024-02-10T07:53:44.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:44 smithi070 ceph-mon[29548]: osdmap e715: 8 total, 8 up, 8 in 2024-02-10T07:53:44.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:44 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:44 smithi070 ceph-mon[39923]: osdmap e715: 8 total, 8 up, 8 in 2024-02-10T07:53:44.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:44 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:44 smithi155 ceph-mon[32688]: osdmap e715: 8 total, 8 up, 8 in 2024-02-10T07:53:45.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:44 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:53:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[39923]: pgmap v1098: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[39923]: osdmap e716: 8 total, 8 up, 8 in 2024-02-10T07:53:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[29548]: pgmap v1098: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[29548]: osdmap e716: 8 total, 8 up, 8 in 2024-02-10T07:53:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:45 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:45 smithi155 ceph-mon[32688]: pgmap v1098: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:53:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:45 smithi155 ceph-mon[32688]: osdmap e716: 8 total, 8 up, 8 in 2024-02-10T07:53:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:45 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:53:46.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:46.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:46 smithi070 ceph-mon[29548]: osdmap e717: 8 total, 8 up, 8 in 2024-02-10T07:53:46.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:46 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:53:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:46 smithi070 ceph-mon[39923]: osdmap e717: 8 total, 8 up, 8 in 2024-02-10T07:53:46.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:46 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:53:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:53:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:46 smithi155 ceph-mon[32688]: osdmap e717: 8 total, 8 up, 8 in 2024-02-10T07:53:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:46 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:53:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[29548]: pgmap v1101: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-144"}]': finished 2024-02-10T07:53:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[29548]: osdmap e718: 8 total, 8 up, 8 in 2024-02-10T07:53:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-144", "mode": "readproxy"}]: dispatch 2024-02-10T07:53:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[39923]: pgmap v1101: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-144"}]': finished 2024-02-10T07:53:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[39923]: osdmap e718: 8 total, 8 up, 8 in 2024-02-10T07:53:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:47 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-144", "mode": "readproxy"}]: dispatch 2024-02-10T07:53:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:47 smithi155 ceph-mon[32688]: pgmap v1101: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T07:53:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-144"}]': finished 2024-02-10T07:53:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:47 smithi155 ceph-mon[32688]: osdmap e718: 8 total, 8 up, 8 in 2024-02-10T07:53:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:47 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-144", "mode": "readproxy"}]: dispatch 2024-02-10T07:53:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:48 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:53:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:48 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-144", "mode": "readproxy"}]': finished 2024-02-10T07:53:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:48 smithi070 ceph-mon[29548]: osdmap e719: 8 total, 8 up, 8 in 2024-02-10T07:53:48.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:48 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:53:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:48 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-144", "mode": "readproxy"}]': finished 2024-02-10T07:53:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:48 smithi070 ceph-mon[39923]: osdmap e719: 8 total, 8 up, 8 in 2024-02-10T07:53:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:48 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:53:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:48 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-144", "mode": "readproxy"}]': finished 2024-02-10T07:53:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:48 smithi155 ceph-mon[32688]: osdmap e719: 8 total, 8 up, 8 in 2024-02-10T07:53:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:49 smithi070 ceph-mon[29548]: pgmap v1104: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:53:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:49 smithi070 ceph-mon[39923]: pgmap v1104: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:53:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:49 smithi155 ceph-mon[32688]: pgmap v1104: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:53:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:53:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:53:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:50.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:53:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:51 smithi070 ceph-mon[29548]: pgmap v1105: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 186 B/s wr, 1 op/s 2024-02-10T07:53:51.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:51 smithi070 ceph-mon[39923]: pgmap v1105: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 186 B/s wr, 1 op/s 2024-02-10T07:53:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:51 smithi155 ceph-mon[32688]: pgmap v1105: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 186 B/s wr, 1 op/s 2024-02-10T07:53:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:53:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:53:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:53:52.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:52 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:52.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:52 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:53.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:52 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:53:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:53 smithi070 ceph-mon[39923]: pgmap v1106: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 157 B/s wr, 1 op/s 2024-02-10T07:53:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:53 smithi070 ceph-mon[29548]: pgmap v1106: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 157 B/s wr, 1 op/s 2024-02-10T07:53:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:53 smithi155 ceph-mon[32688]: pgmap v1106: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 157 B/s wr, 1 op/s 2024-02-10T07:53:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:55 smithi070 ceph-mon[39923]: pgmap v1107: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-10T07:53:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:55 smithi070 ceph-mon[29548]: pgmap v1107: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-10T07:53:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:55 smithi155 ceph-mon[32688]: pgmap v1107: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-10T07:53:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:56 smithi070 ceph-mon[29548]: pgmap v1108: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 B/s wr, 1 op/s 2024-02-10T07:53:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:56 smithi070 ceph-mon[39923]: pgmap v1108: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 B/s wr, 1 op/s 2024-02-10T07:53:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:53:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:53:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:56 smithi155 ceph-mon[32688]: pgmap v1108: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 107 B/s wr, 1 op/s 2024-02-10T07:53:58.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:58.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:58.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:53:59.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:53:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:53:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:59 smithi155 ceph-mon[32688]: pgmap v1109: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 97 B/s wr, 1 op/s 2024-02-10T07:53:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:53:59.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:59 smithi155 ceph-mon[32688]: osdmap e720: 8 total, 8 up, 8 in 2024-02-10T07:53:59.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:53:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:53:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[29548]: pgmap v1109: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 97 B/s wr, 1 op/s 2024-02-10T07:53:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:53:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[29548]: osdmap e720: 8 total, 8 up, 8 in 2024-02-10T07:53:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:53:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[39923]: pgmap v1109: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 97 B/s wr, 1 op/s 2024-02-10T07:53:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:53:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[39923]: osdmap e720: 8 total, 8 up, 8 in 2024-02-10T07:53:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:53:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:54:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:00 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:54:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]': finished 2024-02-10T07:54:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:00 smithi155 ceph-mon[32688]: osdmap e721: 8 total, 8 up, 8 in 2024-02-10T07:54:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:00.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:54:00.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:54:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]': finished 2024-02-10T07:54:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[39923]: osdmap e721: 8 total, 8 up, 8 in 2024-02-10T07:54:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]': finished 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[29548]: osdmap e721: 8 total, 8 up, 8 in 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-144"}]: dispatch 2024-02-10T07:54:00.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:01 smithi155 ceph-mon[32688]: pgmap v1112: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:01 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:01 smithi155 ceph-mon[32688]: osdmap e722: 8 total, 8 up, 8 in 2024-02-10T07:54:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:01 smithi070 ceph-mon[39923]: pgmap v1112: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:01 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:01 smithi070 ceph-mon[39923]: osdmap e722: 8 total, 8 up, 8 in 2024-02-10T07:54:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:01 smithi070 ceph-mon[29548]: pgmap v1112: 332 pgs: 332 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:01 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:01 smithi070 ceph-mon[29548]: osdmap e722: 8 total, 8 up, 8 in 2024-02-10T07:54:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:54:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:54:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:54:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:02 smithi070 ceph-mon[29548]: osdmap e723: 8 total, 8 up, 8 in 2024-02-10T07:54:02.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:02 smithi070 ceph-mon[39923]: osdmap e723: 8 total, 8 up, 8 in 2024-02-10T07:54:02.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:02 smithi155 ceph-mon[32688]: osdmap e723: 8 total, 8 up, 8 in 2024-02-10T07:54:03.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[29548]: pgmap v1115: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[29548]: osdmap e724: 8 total, 8 up, 8 in 2024-02-10T07:54:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:54:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[39923]: pgmap v1115: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[39923]: osdmap e724: 8 total, 8 up, 8 in 2024-02-10T07:54:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:54:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:03 smithi155 ceph-mon[32688]: pgmap v1115: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:03 smithi155 ceph-mon[32688]: osdmap e724: 8 total, 8 up, 8 in 2024-02-10T07:54:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:54:04.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:54:05.016 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[29548]: osdmap e725: 8 total, 8 up, 8 in 2024-02-10T07:54:05.016 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:05.016 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-146"}]': finished 2024-02-10T07:54:05.017 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:54:05.017 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[39923]: osdmap e725: 8 total, 8 up, 8 in 2024-02-10T07:54:05.017 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:05.017 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:04 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-146"}]': finished 2024-02-10T07:54:05.017 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:54:05.017 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:04 smithi155 ceph-mon[32688]: osdmap e725: 8 total, 8 up, 8 in 2024-02-10T07:54:05.018 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:05.018 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:04 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi070-77569-111", "overlaypool": "test-rados-api-smithi070-77569-146"}]': finished 2024-02-10T07:54:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[29548]: pgmap v1118: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:54:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[29548]: osdmap e726: 8 total, 8 up, 8 in 2024-02-10T07:54:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-146", "mode": "writeback"}]: dispatch 2024-02-10T07:54:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:54:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-146", "mode": "writeback"}]': finished 2024-02-10T07:54:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[39923]: pgmap v1118: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:54:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[39923]: osdmap e726: 8 total, 8 up, 8 in 2024-02-10T07:54:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-146", "mode": "writeback"}]: dispatch 2024-02-10T07:54:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:54:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:05 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-146", "mode": "writeback"}]': finished 2024-02-10T07:54:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:05 smithi155 ceph-mon[32688]: pgmap v1118: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:54:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:05 smithi155 ceph-mon[32688]: osdmap e726: 8 total, 8 up, 8 in 2024-02-10T07:54:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-146", "mode": "writeback"}]: dispatch 2024-02-10T07:54:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:05 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T07:54:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:05 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi070-77569-146", "mode": "writeback"}]': finished 2024-02-10T07:54:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:06 smithi070 ceph-mon[29548]: osdmap e727: 8 total, 8 up, 8 in 2024-02-10T07:54:06.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:06 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:54:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:06 smithi070 ceph-mon[39923]: osdmap e727: 8 total, 8 up, 8 in 2024-02-10T07:54:06.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:06 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:54:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:06 smithi155 ceph-mon[32688]: osdmap e727: 8 total, 8 up, 8 in 2024-02-10T07:54:07.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:06 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T07:54:07.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:07 smithi155 ceph-mon[32688]: pgmap v1121: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:54:07.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:54:07.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:07 smithi155 ceph-mon[32688]: osdmap e728: 8 total, 8 up, 8 in 2024-02-10T07:54:07.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:07 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:54:07.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:07 smithi155 ceph-mon[32688]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[29548]: pgmap v1121: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:54:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:54:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[29548]: osdmap e728: 8 total, 8 up, 8 in 2024-02-10T07:54:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:54:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[29548]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[39923]: pgmap v1121: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T07:54:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_count","val": "2"}]': finished 2024-02-10T07:54:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[39923]: osdmap e728: 8 total, 8 up, 8 in 2024-02-10T07:54:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T07:54:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:07 smithi070 ceph-mon[39923]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:54:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[39923]: osdmap e729: 8 total, 8 up, 8 in 2024-02-10T07:54:08.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:54:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:54:08.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:54:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:54:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[29548]: osdmap e729: 8 total, 8 up, 8 in 2024-02-10T07:54:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:54:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:54:08.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:08 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:54:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_period","val": "600"}]': finished 2024-02-10T07:54:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:08 smithi155 ceph-mon[32688]: osdmap e729: 8 total, 8 up, 8 in 2024-02-10T07:54:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T07:54:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:08 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T07:54:09.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:08 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T07:54:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[29548]: pgmap v1124: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:54:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[29548]: osdmap e730: 8 total, 8 up, 8 in 2024-02-10T07:54:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[29548]: osdmap e731: 8 total, 8 up, 8 in 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[39923]: pgmap v1124: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[39923]: osdmap e730: 8 total, 8 up, 8 in 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[39923]: osdmap e731: 8 total, 8 up, 8 in 2024-02-10T07:54:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:09 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:54:10.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:54:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:54:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:09 smithi155 ceph-mon[32688]: pgmap v1124: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 715 GiB / 715 GiB avail 2024-02-10T07:54:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:09 smithi155 ceph-mon[32688]: osdmap e730: 8 total, 8 up, 8 in 2024-02-10T07:54:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T07:54:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T07:54:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:09 smithi155 ceph-mon[32688]: osdmap e731: 8 total, 8 up, 8 in 2024-02-10T07:54:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:09 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T07:54:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:10 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:54:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:10 smithi070 ceph-mon[29548]: osdmap e732: 8 total, 8 up, 8 in 2024-02-10T07:54:10.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:10.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:10 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:54:10.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:10 smithi070 ceph-mon[39923]: osdmap e732: 8 total, 8 up, 8 in 2024-02-10T07:54:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:10 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-146","var": "target_max_objects","val": "1"}]': finished 2024-02-10T07:54:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:10 smithi155 ceph-mon[32688]: osdmap e732: 8 total, 8 up, 8 in 2024-02-10T07:54:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:11 smithi070 ceph-mon[29548]: pgmap v1127: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:11 smithi070 ceph-mon[39923]: pgmap v1127: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:11 smithi155 ceph-mon[32688]: pgmap v1127: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:54:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:54:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:54:13.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:13 smithi070 ceph-mon[29548]: pgmap v1129: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:13.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:13 smithi070 ceph-mon[29548]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:54:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:13 smithi070 ceph-mon[39923]: pgmap v1129: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:13.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:13 smithi070 ceph-mon[39923]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:54:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:13 smithi155 ceph-mon[32688]: pgmap v1129: 332 pgs: 332 active+clean; 4.4 MiB data, 736 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:13.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:13 smithi155 ceph-mon[32688]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T07:54:15.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:15 smithi070 ceph-mon[29548]: pgmap v1130: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:15.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:15 smithi070 ceph-mon[39923]: pgmap v1130: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:15.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:15 smithi155 ceph-mon[32688]: pgmap v1130: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-10T07:54:17.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:17 smithi155 ceph-mon[32688]: pgmap v1131: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:54:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:17 smithi070 ceph-mon[29548]: pgmap v1131: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:54:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:17 smithi070 ceph-mon[39923]: pgmap v1131: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T07:54:19.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:54:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:54:19.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:19 smithi155 ceph-mon[32688]: pgmap v1132: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 599 B/s rd, 0 B/s wr, 0 op/s 2024-02-10T07:54:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:19 smithi070 ceph-mon[29548]: pgmap v1132: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 599 B/s rd, 0 B/s wr, 0 op/s 2024-02-10T07:54:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:19 smithi070 ceph-mon[39923]: pgmap v1132: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 599 B/s rd, 0 B/s wr, 0 op/s 2024-02-10T07:54:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:21 smithi070 ceph-mon[29548]: pgmap v1133: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:54:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:21 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:21 smithi070 ceph-mon[39923]: pgmap v1133: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:54:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:21 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:21 smithi155 ceph-mon[32688]: pgmap v1133: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:54:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:21 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:54:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:54:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[29548]: osdmap e733: 8 total, 8 up, 8 in 2024-02-10T07:54:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]': finished 2024-02-10T07:54:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[39923]: osdmap e733: 8 total, 8 up, 8 in 2024-02-10T07:54:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:22 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]': finished 2024-02-10T07:54:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:22 smithi155 ceph-mon[32688]: osdmap e733: 8 total, 8 up, 8 in 2024-02-10T07:54:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:23.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:22 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]': finished 2024-02-10T07:54:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[29548]: pgmap v1135: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:54:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[29548]: osdmap e734: 8 total, 8 up, 8 in 2024-02-10T07:54:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[39923]: pgmap v1135: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:54:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[39923]: osdmap e734: 8 total, 8 up, 8 in 2024-02-10T07:54:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:23 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:23 smithi155 ceph-mon[32688]: pgmap v1135: 332 pgs: 332 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-10T07:54:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:23 smithi155 ceph-mon[32688]: osdmap e734: 8 total, 8 up, 8 in 2024-02-10T07:54:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:23 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-146"}]: dispatch 2024-02-10T07:54:24.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:24 smithi070 ceph-mon[29548]: osdmap e735: 8 total, 8 up, 8 in 2024-02-10T07:54:24.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:24 smithi070 ceph-mon[39923]: osdmap e735: 8 total, 8 up, 8 in 2024-02-10T07:54:25.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:24 smithi155 ceph-mon[32688]: osdmap e735: 8 total, 8 up, 8 in 2024-02-10T07:54:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[39923]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:54:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[39923]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:54:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[39923]: osdmap e736: 8 total, 8 up, 8 in 2024-02-10T07:54:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[29548]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:54:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[29548]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:54:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[29548]: osdmap e736: 8 total, 8 up, 8 in 2024-02-10T07:54:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:25 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:26.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:25 smithi155 ceph-mon[32688]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:54:26.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:25 smithi155 ceph-mon[32688]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T07:54:26.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:25 smithi155 ceph-mon[32688]: osdmap e736: 8 total, 8 up, 8 in 2024-02-10T07:54:26.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:25 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[29548]: osdmap e737: 8 total, 8 up, 8 in 2024-02-10T07:54:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:54:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[39923]: osdmap e737: 8 total, 8 up, 8 in 2024-02-10T07:54:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:54:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:26 smithi155 ceph-mon[32688]: osdmap e737: 8 total, 8 up, 8 in 2024-02-10T07:54:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:26 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T07:54:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[29548]: pgmap v1141: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:54:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:54:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[29548]: osdmap e738: 8 total, 8 up, 8 in 2024-02-10T07:54:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]: dispatch 2024-02-10T07:54:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[39923]: pgmap v1141: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:54:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:54:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[39923]: osdmap e738: 8 total, 8 up, 8 in 2024-02-10T07:54:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:27 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]: dispatch 2024-02-10T07:54:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:27 smithi155 ceph-mon[32688]: pgmap v1141: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T07:54:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T07:54:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:27 smithi155 ceph-mon[32688]: osdmap e738: 8 total, 8 up, 8 in 2024-02-10T07:54:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:27 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]: dispatch 2024-02-10T07:54:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]': finished 2024-02-10T07:54:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[29548]: osdmap e739: 8 total, 8 up, 8 in 2024-02-10T07:54:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]: dispatch 2024-02-10T07:54:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]': finished 2024-02-10T07:54:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[39923]: osdmap e739: 8 total, 8 up, 8 in 2024-02-10T07:54:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:28.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:28 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]: dispatch 2024-02-10T07:54:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]': finished 2024-02-10T07:54:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:28 smithi155 ceph-mon[32688]: osdmap e739: 8 total, 8 up, 8 in 2024-02-10T07:54:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:28 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-148"}]: dispatch 2024-02-10T07:54:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:29 smithi070 ceph-mon[29548]: pgmap v1144: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:29 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:29 smithi070 ceph-mon[29548]: osdmap e740: 8 total, 8 up, 8 in 2024-02-10T07:54:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:29 smithi070 ceph-mon[39923]: pgmap v1144: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:29 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:29 smithi070 ceph-mon[39923]: osdmap e740: 8 total, 8 up, 8 in 2024-02-10T07:54:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:54:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:54:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:29 smithi155 ceph-mon[32688]: pgmap v1144: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:29 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:29 smithi155 ceph-mon[32688]: osdmap e740: 8 total, 8 up, 8 in 2024-02-10T07:54:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:30 smithi070 ceph-mon[29548]: osdmap e741: 8 total, 8 up, 8 in 2024-02-10T07:54:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:30 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:30.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:30 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:30 smithi070 ceph-mon[39923]: osdmap e741: 8 total, 8 up, 8 in 2024-02-10T07:54:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:30 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:30.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:30 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:30 smithi155 ceph-mon[32688]: osdmap e741: 8 total, 8 up, 8 in 2024-02-10T07:54:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:30 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:31.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:30 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:32.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[29548]: pgmap v1147: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:54:32.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[29548]: osdmap e742: 8 total, 8 up, 8 in 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-150"}]: dispatch 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[39923]: pgmap v1147: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[39923]: osdmap e742: 8 total, 8 up, 8 in 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:31 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-150"}]: dispatch 2024-02-10T07:54:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:31 smithi155 ceph-mon[32688]: pgmap v1147: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:54:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:31 smithi155 ceph-mon[32688]: osdmap e742: 8 total, 8 up, 8 in 2024-02-10T07:54:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:31 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-150"}]: dispatch 2024-02-10T07:54:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:54:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:54:33.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[39923]: osdmap e743: 8 total, 8 up, 8 in 2024-02-10T07:54:33.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[39923]: pgmap v1150: 300 pgs: 300 active+clean; 4.4 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:54:33.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[39923]: osdmap e744: 8 total, 8 up, 8 in 2024-02-10T07:54:33.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[29548]: osdmap e743: 8 total, 8 up, 8 in 2024-02-10T07:54:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[29548]: pgmap v1150: 300 pgs: 300 active+clean; 4.4 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:54:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[29548]: osdmap e744: 8 total, 8 up, 8 in 2024-02-10T07:54:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:32 smithi155 ceph-mon[32688]: osdmap e743: 8 total, 8 up, 8 in 2024-02-10T07:54:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:32 smithi155 ceph-mon[32688]: pgmap v1150: 300 pgs: 300 active+clean; 4.4 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T07:54:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:32 smithi155 ceph-mon[32688]: osdmap e744: 8 total, 8 up, 8 in 2024-02-10T07:54:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[29548]: osdmap e745: 8 total, 8 up, 8 in 2024-02-10T07:54:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:34.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-152"}]: dispatch 2024-02-10T07:54:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[39923]: osdmap e745: 8 total, 8 up, 8 in 2024-02-10T07:54:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:34.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-152"}]: dispatch 2024-02-10T07:54:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:34 smithi155 ceph-mon[32688]: osdmap e745: 8 total, 8 up, 8 in 2024-02-10T07:54:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:35.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-152"}]: dispatch 2024-02-10T07:54:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[29548]: pgmap v1153: 332 pgs: 332 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T07:54:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[29548]: osdmap e746: 8 total, 8 up, 8 in 2024-02-10T07:54:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:54:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[39923]: pgmap v1153: 332 pgs: 332 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T07:54:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[39923]: osdmap e746: 8 total, 8 up, 8 in 2024-02-10T07:54:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:54:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:35 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:35 smithi155 ceph-mon[32688]: pgmap v1153: 332 pgs: 332 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T07:54:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:35 smithi155 ceph-mon[32688]: osdmap e746: 8 total, 8 up, 8 in 2024-02-10T07:54:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:35 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:54:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:35 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:35 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[39923]: osdmap e747: 8 total, 8 up, 8 in 2024-02-10T07:54:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:54:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:54:36.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[29548]: osdmap e747: 8 total, 8 up, 8 in 2024-02-10T07:54:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:54:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:54:36.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:36 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:36 smithi155 ceph-mon[32688]: osdmap e747: 8 total, 8 up, 8 in 2024-02-10T07:54:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:54:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:36 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:54:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:36 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:54:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:36 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:54:37.473 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.473 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.474 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.475 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.476 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.477 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.478 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.479 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.480 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.481 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.481 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.481 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:54:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:37 smithi155 ceph-mon[32688]: pgmap v1156: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:54:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:37 smithi155 ceph-mon[32688]: osdmap e748: 8 total, 8 up, 8 in 2024-02-10T07:54:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-111","var": "dedup_tier","val": "test-rados-api-smithi070-77569-154"}]: dispatch 2024-02-10T07:54:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-154"}]: dispatch 2024-02-10T07:54:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:37 smithi155 ceph-mon[32688]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[29548]: pgmap v1156: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:54:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[29548]: osdmap e748: 8 total, 8 up, 8 in 2024-02-10T07:54:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-111","var": "dedup_tier","val": "test-rados-api-smithi070-77569-154"}]: dispatch 2024-02-10T07:54:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-154"}]: dispatch 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[29548]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[39923]: pgmap v1156: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi070-77569-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[39923]: osdmap e748: 8 total, 8 up, 8 in 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi070-77569-111","var": "dedup_tier","val": "test-rados-api-smithi070-77569-154"}]: dispatch 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi070-77569-111", "tierpool": "test-rados-api-smithi070-77569-154"}]: dispatch 2024-02-10T07:54:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:37 smithi070 ceph-mon[39923]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:38 smithi070 ceph-mon[29548]: osdmap e749: 8 total, 8 up, 8 in 2024-02-10T07:54:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:38 smithi070 ceph-mon[39923]: osdmap e749: 8 total, 8 up, 8 in 2024-02-10T07:54:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:38 smithi155 ceph-mon[32688]: osdmap e749: 8 total, 8 up, 8 in 2024-02-10T07:54:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:39 smithi070 ceph-mon[39923]: pgmap v1159: 300 pgs: 300 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:39 smithi070 ceph-mon[39923]: osdmap e750: 8 total, 8 up, 8 in 2024-02-10T07:54:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:39 smithi070 ceph-mon[29548]: pgmap v1159: 300 pgs: 300 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:39 smithi070 ceph-mon[29548]: osdmap e750: 8 total, 8 up, 8 in 2024-02-10T07:54:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:40.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:54:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:54:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:39 smithi155 ceph-mon[32688]: pgmap v1159: 300 pgs: 300 active+clean; 4.4 MiB data, 782 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:39 smithi155 ceph-mon[32688]: osdmap e750: 8 total, 8 up, 8 in 2024-02-10T07:54:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:40.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: wait LibRadosTwoPoolsECPP.TryFlush (7867 ms) 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13019 ms) 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8039 ms) 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13097 ms) 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7874 ms) 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-10T07:54:40.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8040 ms) 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: ok, hit_set contains 325:602f83fe:::foo:head 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9051 ms) 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: first is 1707551594 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,1707551599,1707551600,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,1707551599,1707551600,0 2024-02-10T07:54:40.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,1707551599,1707551600,0 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,1707551599,1707551600,1707551602,1707551603,0 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,1707551599,1707551600,1707551602,1707551603,0 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551594,1707551596,1707551597,1707551599,1707551600,1707551602,1707551603,0 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: got ls 1707551597,1707551599,1707551600,1707551602,1707551603,1707551605,1707551606,0 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: first now 1707551597, trimmed 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20907 ms) 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: foo0 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (15066 ms) 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17949 ms) 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-10T07:54:40.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23015 ms) 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5028 ms) 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3011 ms) 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3010 ms) 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3020 ms) 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (233752 ms total) 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (862639 ms total) 2024-02-10T07:54:40.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77238 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77238 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77514 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77514 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78036 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78036 2024-02-10T07:54:40.613 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77794 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77794 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77627 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77627 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77317 2024-02-10T07:54:40.614 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77317 2024-02-10T07:54:40.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:40.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:40 smithi155 ceph-mon[32688]: osdmap e751: 8 total, 8 up, 8 in 2024-02-10T07:54:40.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:40.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:40.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:40.756 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:54:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=sqlstore.transactions t=2024-02-10T07:54:40.380200837Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-10T07:54:40.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:40.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[39923]: osdmap e751: 8 total, 8 up, 8 in 2024-02-10T07:54:40.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:40.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[29548]: osdmap e751: 8 total, 8 up, 8 in 2024-02-10T07:54:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]: dispatch 2024-02-10T07:54:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/480905714' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi070-77569-111"}]': finished 2024-02-10T07:54:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:41 smithi070 ceph-mon[29548]: pgmap v1162: 292 pgs: 292 active+clean; 582 KiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:41 smithi070 ceph-mon[29548]: osdmap e752: 8 total, 8 up, 8 in 2024-02-10T07:54:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:41 smithi070 ceph-mon[39923]: pgmap v1162: 292 pgs: 292 active+clean; 582 KiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:41 smithi070 ceph-mon[39923]: osdmap e752: 8 total, 8 up, 8 in 2024-02-10T07:54:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:41 smithi155 ceph-mon[32688]: pgmap v1162: 292 pgs: 292 active+clean; 582 KiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:41 smithi155 ceph-mon[32688]: osdmap e752: 8 total, 8 up, 8 in 2024-02-10T07:54:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:54:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:54:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:54:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:42 smithi070 ceph-mon[29548]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:42.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:42 smithi070 ceph-mon[39923]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:42 smithi155 ceph-mon[32688]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T07:54:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:43 smithi070 ceph-mon[29548]: pgmap v1164: 292 pgs: 292 active+clean; 582 KiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:43 smithi070 ceph-mon[39923]: pgmap v1164: 292 pgs: 292 active+clean; 582 KiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:43 smithi155 ceph-mon[32688]: pgmap v1164: 292 pgs: 292 active+clean; 582 KiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:45 smithi070 ceph-mon[29548]: pgmap v1165: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:45 smithi070 ceph-mon[39923]: pgmap v1165: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:45 smithi155 ceph-mon[32688]: pgmap v1165: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail 2024-02-10T07:54:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:47 smithi070 ceph-mon[29548]: pgmap v1166: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T07:54:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:47 smithi070 ceph-mon[39923]: pgmap v1166: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T07:54:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:47 smithi155 ceph-mon[32688]: pgmap v1166: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T07:54:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:49 smithi070 ceph-mon[29548]: pgmap v1167: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 602 B/s rd, 0 op/s 2024-02-10T07:54:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:49 smithi070 ceph-mon[39923]: pgmap v1167: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 602 B/s rd, 0 op/s 2024-02-10T07:54:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:49 smithi155 ceph-mon[32688]: pgmap v1167: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 602 B/s rd, 0 op/s 2024-02-10T07:54:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:54:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:54:50.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:50.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:51.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:51 smithi070 ceph-mon[29548]: pgmap v1168: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:54:51.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:51 smithi070 ceph-mon[39923]: pgmap v1168: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:54:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:51 smithi155 ceph-mon[32688]: pgmap v1168: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T07:54:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:54:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:54:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:54:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:53 smithi070 ceph-mon[29548]: pgmap v1169: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-02-10T07:54:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:53 smithi070 ceph-mon[39923]: pgmap v1169: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-02-10T07:54:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:53 smithi155 ceph-mon[32688]: pgmap v1169: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-02-10T07:54:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:55 smithi070 ceph-mon[29548]: pgmap v1170: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:55 smithi070 ceph-mon[39923]: pgmap v1170: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:55 smithi155 ceph-mon[32688]: pgmap v1170: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:54:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:54:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:57 smithi070 ceph-mon[39923]: pgmap v1171: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:54:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:57 smithi070 ceph-mon[29548]: pgmap v1171: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:54:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:57 smithi155 ceph-mon[32688]: pgmap v1171: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:54:58.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:58 smithi070 ceph-mon[39923]: pgmap v1172: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:54:58.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:58 smithi070 ceph-mon[29548]: pgmap v1172: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:54:59.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:58 smithi155 ceph-mon[32688]: pgmap v1172: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:54:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:54:59 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:54:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:54:59 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:00.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:54:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:55:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:54:59 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:00 smithi070 ceph-mon[39923]: pgmap v1173: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:00.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:00 smithi070 ceph-mon[29548]: pgmap v1173: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:00 smithi155 ceph-mon[32688]: pgmap v1173: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:55:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:55:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:55:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:03 smithi070 ceph-mon[39923]: pgmap v1174: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:03 smithi070 ceph-mon[29548]: pgmap v1174: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:03 smithi155 ceph-mon[32688]: pgmap v1174: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:05 smithi070 ceph-mon[39923]: pgmap v1175: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:05 smithi070 ceph-mon[29548]: pgmap v1175: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:05 smithi155 ceph-mon[32688]: pgmap v1175: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:07.807 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:07 smithi155 ceph-mon[32688]: pgmap v1176: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:07 smithi070 ceph-mon[29548]: pgmap v1176: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:07 smithi070 ceph-mon[39923]: pgmap v1176: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:09 smithi070 ceph-mon[29548]: pgmap v1177: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:09 smithi070 ceph-mon[39923]: pgmap v1177: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:09 smithi155 ceph-mon[32688]: pgmap v1177: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:10.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:55:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:55:10.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:10 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:10.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:10 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:11.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:10 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:11 smithi070 ceph-mon[29548]: pgmap v1178: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:11 smithi070 ceph-mon[39923]: pgmap v1178: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:11 smithi155 ceph-mon[32688]: pgmap v1178: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:55:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:55:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:55:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:13 smithi070 ceph-mon[29548]: pgmap v1179: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:13 smithi070 ceph-mon[39923]: pgmap v1179: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:13 smithi155 ceph-mon[32688]: pgmap v1179: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:15 smithi070 ceph-mon[29548]: pgmap v1180: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:15 smithi070 ceph-mon[39923]: pgmap v1180: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:15 smithi155 ceph-mon[32688]: pgmap v1180: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:17 smithi070 ceph-mon[29548]: pgmap v1181: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:17 smithi070 ceph-mon[39923]: pgmap v1181: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:17 smithi155 ceph-mon[32688]: pgmap v1181: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:19 smithi070 ceph-mon[29548]: pgmap v1182: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:19 smithi070 ceph-mon[39923]: pgmap v1182: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:19 smithi155 ceph-mon[32688]: pgmap v1182: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:20.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:55:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:55:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:21 smithi070 ceph-mon[29548]: pgmap v1183: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:21 smithi070 ceph-mon[39923]: pgmap v1183: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:21 smithi155 ceph-mon[32688]: pgmap v1183: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:55:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:55:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:55:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:23 smithi070 ceph-mon[39923]: pgmap v1184: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:23 smithi070 ceph-mon[29548]: pgmap v1184: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:23 smithi155 ceph-mon[32688]: pgmap v1184: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:25 smithi070 ceph-mon[29548]: pgmap v1185: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:25 smithi070 ceph-mon[39923]: pgmap v1185: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:25 smithi155 ceph-mon[32688]: pgmap v1185: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:27 smithi070 ceph-mon[29548]: pgmap v1186: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:27 smithi070 ceph-mon[39923]: pgmap v1186: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:27 smithi155 ceph-mon[32688]: pgmap v1186: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:29 smithi070 ceph-mon[29548]: pgmap v1187: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:29 smithi070 ceph-mon[39923]: pgmap v1187: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:55:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:55:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:29 smithi155 ceph-mon[32688]: pgmap v1187: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:31 smithi070 ceph-mon[29548]: pgmap v1188: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:31 smithi070 ceph-mon[39923]: pgmap v1188: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:31 smithi155 ceph-mon[32688]: pgmap v1188: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:55:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:55:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:55:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:33 smithi070 ceph-mon[29548]: pgmap v1189: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:33 smithi070 ceph-mon[39923]: pgmap v1189: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:33 smithi155 ceph-mon[32688]: pgmap v1189: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:35 smithi070 ceph-mon[29548]: pgmap v1190: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:35 smithi070 ceph-mon[39923]: pgmap v1190: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:35 smithi155 ceph-mon[32688]: pgmap v1190: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:36.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:36 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:55:36.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:36 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:55:37.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:36 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:55:37.906 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:37 smithi155 ceph-mon[32688]: pgmap v1191: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:55:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:55:37.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:55:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[29548]: pgmap v1191: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:55:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:55:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:55:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[39923]: pgmap v1191: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:55:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:55:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:55:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:39 smithi070 ceph-mon[29548]: pgmap v1192: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:39 smithi070 ceph-mon[39923]: pgmap v1192: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:39 smithi155 ceph-mon[32688]: pgmap v1192: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:40.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:55:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:55:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:41 smithi070 ceph-mon[29548]: pgmap v1193: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:41 smithi070 ceph-mon[39923]: pgmap v1193: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:41 smithi155 ceph-mon[32688]: pgmap v1193: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:55:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:55:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:55:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:43 smithi070 ceph-mon[29548]: pgmap v1194: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:43 smithi070 ceph-mon[39923]: pgmap v1194: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:43 smithi155 ceph-mon[32688]: pgmap v1194: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:45 smithi070 ceph-mon[29548]: pgmap v1195: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:45 smithi070 ceph-mon[39923]: pgmap v1195: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:45 smithi155 ceph-mon[32688]: pgmap v1195: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:47 smithi070 ceph-mon[29548]: pgmap v1196: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:47 smithi070 ceph-mon[39923]: pgmap v1196: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:47 smithi155 ceph-mon[32688]: pgmap v1196: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:49 smithi070 ceph-mon[29548]: pgmap v1197: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:49 smithi070 ceph-mon[39923]: pgmap v1197: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:49 smithi155 ceph-mon[32688]: pgmap v1197: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:55:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:55:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:51 smithi070 ceph-mon[29548]: pgmap v1198: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:51 smithi070 ceph-mon[39923]: pgmap v1198: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:55:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:51 smithi155 ceph-mon[32688]: pgmap v1198: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:55:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:55:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:55:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:53 smithi070 ceph-mon[39923]: pgmap v1199: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:53 smithi070 ceph-mon[29548]: pgmap v1199: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:53 smithi155 ceph-mon[32688]: pgmap v1199: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:55 smithi070 ceph-mon[29548]: pgmap v1200: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:55 smithi070 ceph-mon[39923]: pgmap v1200: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:55 smithi155 ceph-mon[32688]: pgmap v1200: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:55:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:56.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:55:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:57 smithi070 ceph-mon[29548]: pgmap v1201: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:57 smithi070 ceph-mon[39923]: pgmap v1201: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:57 smithi155 ceph-mon[32688]: pgmap v1201: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:55:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:55:59 smithi070 ceph-mon[29548]: pgmap v1202: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:55:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:55:59 smithi070 ceph-mon[39923]: pgmap v1202: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:00.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:55:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:56:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:55:59 smithi155 ceph-mon[32688]: pgmap v1202: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:01 smithi070 ceph-mon[39923]: pgmap v1203: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:01 smithi070 ceph-mon[29548]: pgmap v1203: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:01 smithi155 ceph-mon[32688]: pgmap v1203: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:56:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:56:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:56:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:03 smithi070 ceph-mon[29548]: pgmap v1204: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:03 smithi070 ceph-mon[39923]: pgmap v1204: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:03 smithi155 ceph-mon[32688]: pgmap v1204: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:05 smithi070 ceph-mon[29548]: pgmap v1205: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:05 smithi070 ceph-mon[39923]: pgmap v1205: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:05 smithi155 ceph-mon[32688]: pgmap v1205: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:07 smithi070 ceph-mon[29548]: pgmap v1206: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:07 smithi070 ceph-mon[39923]: pgmap v1206: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:08.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:07 smithi155 ceph-mon[32688]: pgmap v1206: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:09 smithi070 ceph-mon[29548]: pgmap v1207: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:09 smithi070 ceph-mon[39923]: pgmap v1207: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:10.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:56:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:56:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:09 smithi155 ceph-mon[32688]: pgmap v1207: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:11.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:11 smithi070 ceph-mon[39923]: pgmap v1208: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:11 smithi070 ceph-mon[29548]: pgmap v1208: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:11 smithi155 ceph-mon[32688]: pgmap v1208: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:56:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:56:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:56:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:13 smithi070 ceph-mon[39923]: pgmap v1209: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:13 smithi070 ceph-mon[29548]: pgmap v1209: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:13 smithi155 ceph-mon[32688]: pgmap v1209: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:15 smithi070 ceph-mon[29548]: pgmap v1210: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:15 smithi070 ceph-mon[39923]: pgmap v1210: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:15 smithi155 ceph-mon[32688]: pgmap v1210: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:17.485 INFO:tasks.workunit.client.0.smithi070.stdout:ing 1 sec 2024-02-10T07:56:17.486 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.486 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.486 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.486 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.486 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.486 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.486 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.487 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.488 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.489 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.490 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.491 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.492 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.493 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:56:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:17 smithi070 ceph-mon[39923]: pgmap v1211: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:17 smithi070 ceph-mon[29548]: pgmap v1211: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:17 smithi155 ceph-mon[32688]: pgmap v1211: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:19 smithi070 ceph-mon[39923]: pgmap v1212: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:19 smithi070 ceph-mon[29548]: pgmap v1212: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:20.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:56:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:56:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:19 smithi155 ceph-mon[32688]: pgmap v1212: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:21 smithi070 ceph-mon[29548]: pgmap v1213: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:21 smithi070 ceph-mon[39923]: pgmap v1213: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:21 smithi155 ceph-mon[32688]: pgmap v1213: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:56:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:56:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:56:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:23 smithi070 ceph-mon[29548]: pgmap v1214: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:23 smithi070 ceph-mon[39923]: pgmap v1214: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:23 smithi155 ceph-mon[32688]: pgmap v1214: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:25 smithi070 ceph-mon[29548]: pgmap v1215: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:25 smithi070 ceph-mon[39923]: pgmap v1215: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:25 smithi155 ceph-mon[32688]: pgmap v1215: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:27 smithi070 ceph-mon[29548]: pgmap v1216: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:27 smithi070 ceph-mon[39923]: pgmap v1216: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:27 smithi155 ceph-mon[32688]: pgmap v1216: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:29 smithi070 ceph-mon[29548]: pgmap v1217: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:29 smithi070 ceph-mon[39923]: pgmap v1217: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:29 smithi155 ceph-mon[32688]: pgmap v1217: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:56:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:56:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:31 smithi070 ceph-mon[29548]: pgmap v1218: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:31 smithi070 ceph-mon[39923]: pgmap v1218: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:31 smithi155 ceph-mon[32688]: pgmap v1218: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:56:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:56:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:56:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:33 smithi070 ceph-mon[29548]: pgmap v1219: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:33 smithi070 ceph-mon[39923]: pgmap v1219: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:33 smithi155 ceph-mon[32688]: pgmap v1219: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:35 smithi070 ceph-mon[29548]: pgmap v1220: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:35 smithi070 ceph-mon[39923]: pgmap v1220: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:35 smithi155 ceph-mon[32688]: pgmap v1220: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[29548]: pgmap v1221: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:56:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:56:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:56:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:56:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[39923]: pgmap v1221: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:56:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:56:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:56:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:56:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:37 smithi155 ceph-mon[32688]: pgmap v1221: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:56:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:56:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:56:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:56:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:38 smithi070 ceph-mon[29548]: pgmap v1222: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:38 smithi070 ceph-mon[39923]: pgmap v1222: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:38 smithi155 ceph-mon[32688]: pgmap v1222: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:40.006 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:56:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:56:41.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:41.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:41 smithi070 ceph-mon[39923]: pgmap v1223: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:41.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:41.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:41.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:41 smithi070 ceph-mon[29548]: pgmap v1223: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:41.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:41 smithi155 ceph-mon[32688]: pgmap v1223: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:41.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:56:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:56:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:56:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:43 smithi070 ceph-mon[29548]: pgmap v1224: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:43 smithi070 ceph-mon[39923]: pgmap v1224: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:43 smithi155 ceph-mon[32688]: pgmap v1224: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:45 smithi070 ceph-mon[29548]: pgmap v1225: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:45 smithi070 ceph-mon[39923]: pgmap v1225: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:45 smithi155 ceph-mon[32688]: pgmap v1225: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:47 smithi070 ceph-mon[39923]: pgmap v1226: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:47 smithi070 ceph-mon[29548]: pgmap v1226: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:47 smithi155 ceph-mon[32688]: pgmap v1226: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:49 smithi070 ceph-mon[29548]: pgmap v1227: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:49 smithi070 ceph-mon[39923]: pgmap v1227: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:49 smithi155 ceph-mon[32688]: pgmap v1227: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:56:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:56:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:51 smithi070 ceph-mon[29548]: pgmap v1228: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:51 smithi070 ceph-mon[39923]: pgmap v1228: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:56:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:51 smithi155 ceph-mon[32688]: pgmap v1228: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:56:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:56:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:56:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:53 smithi070 ceph-mon[29548]: pgmap v1229: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:53 smithi070 ceph-mon[39923]: pgmap v1229: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:53 smithi155 ceph-mon[32688]: pgmap v1229: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:55 smithi070 ceph-mon[29548]: pgmap v1230: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:55 smithi070 ceph-mon[39923]: pgmap v1230: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:55 smithi155 ceph-mon[32688]: pgmap v1230: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:56:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:56.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:56:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:57 smithi070 ceph-mon[29548]: pgmap v1231: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:57 smithi070 ceph-mon[39923]: pgmap v1231: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:57 smithi155 ceph-mon[32688]: pgmap v1231: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:56:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:56:59 smithi070 ceph-mon[29548]: pgmap v1232: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:56:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:56:59 smithi070 ceph-mon[39923]: pgmap v1232: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:56:59 smithi155 ceph-mon[32688]: pgmap v1232: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:00.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:56:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:57:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:01 smithi070 ceph-mon[29548]: pgmap v1233: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:01 smithi070 ceph-mon[39923]: pgmap v1233: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:01 smithi155 ceph-mon[32688]: pgmap v1233: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:57:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:57:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:57:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:03 smithi070 ceph-mon[29548]: pgmap v1234: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:03 smithi070 ceph-mon[39923]: pgmap v1234: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:03 smithi155 ceph-mon[32688]: pgmap v1234: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:05 smithi070 ceph-mon[29548]: pgmap v1235: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:05 smithi070 ceph-mon[39923]: pgmap v1235: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:05 smithi155 ceph-mon[32688]: pgmap v1235: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:07 smithi070 ceph-mon[29548]: pgmap v1236: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:07 smithi070 ceph-mon[39923]: pgmap v1236: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:07 smithi155 ceph-mon[32688]: pgmap v1236: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:09 smithi070 ceph-mon[29548]: pgmap v1237: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:09 smithi070 ceph-mon[39923]: pgmap v1237: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:09 smithi155 ceph-mon[32688]: pgmap v1237: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:10.006 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:57:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:57:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:11 smithi070 ceph-mon[29548]: pgmap v1238: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:11 smithi070 ceph-mon[39923]: pgmap v1238: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:11 smithi155 ceph-mon[32688]: pgmap v1238: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:57:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:57:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:57:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:13 smithi070 ceph-mon[29548]: pgmap v1239: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:13 smithi070 ceph-mon[39923]: pgmap v1239: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:13 smithi155 ceph-mon[32688]: pgmap v1239: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:15 smithi070 ceph-mon[29548]: pgmap v1240: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:15 smithi070 ceph-mon[39923]: pgmap v1240: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:15 smithi155 ceph-mon[32688]: pgmap v1240: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:17 smithi070 ceph-mon[29548]: pgmap v1241: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:17 smithi070 ceph-mon[39923]: pgmap v1241: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:17 smithi155 ceph-mon[32688]: pgmap v1241: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:19 smithi070 ceph-mon[29548]: pgmap v1242: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:19 smithi070 ceph-mon[39923]: pgmap v1242: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:19 smithi155 ceph-mon[32688]: pgmap v1242: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:20.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:57:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:57:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:21 smithi070 ceph-mon[29548]: pgmap v1243: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:21 smithi070 ceph-mon[39923]: pgmap v1243: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:21 smithi155 ceph-mon[32688]: pgmap v1243: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:57:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:57:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:57:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:23 smithi070 ceph-mon[29548]: pgmap v1244: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:23 smithi070 ceph-mon[39923]: pgmap v1244: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:23 smithi155 ceph-mon[32688]: pgmap v1244: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:25 smithi070 ceph-mon[39923]: pgmap v1245: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:25.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:25 smithi070 ceph-mon[29548]: pgmap v1245: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:25 smithi155 ceph-mon[32688]: pgmap v1245: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:27 smithi070 ceph-mon[39923]: pgmap v1246: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:27 smithi070 ceph-mon[29548]: pgmap v1246: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:27 smithi155 ceph-mon[32688]: pgmap v1246: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:29 smithi070 ceph-mon[39923]: pgmap v1247: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:29 smithi070 ceph-mon[29548]: pgmap v1247: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:57:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:57:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:29 smithi155 ceph-mon[32688]: pgmap v1247: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:31 smithi070 ceph-mon[39923]: pgmap v1248: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:31 smithi070 ceph-mon[29548]: pgmap v1248: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:31 smithi155 ceph-mon[32688]: pgmap v1248: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:57:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:57:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:57:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:33 smithi070 ceph-mon[29548]: pgmap v1249: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:33 smithi070 ceph-mon[39923]: pgmap v1249: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:33 smithi155 ceph-mon[32688]: pgmap v1249: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:35 smithi070 ceph-mon[29548]: pgmap v1250: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:35 smithi070 ceph-mon[39923]: pgmap v1250: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:35 smithi155 ceph-mon[32688]: pgmap v1250: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:37 smithi070 ceph-mon[29548]: pgmap v1251: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:37 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:57:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:37 smithi070 ceph-mon[39923]: pgmap v1251: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:37 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:57:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:37 smithi155 ceph-mon[32688]: pgmap v1251: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:37 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:57:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:57:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:57:38.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:57:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:57:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:57:38.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:57:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:57:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:57:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:57:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:39 smithi070 ceph-mon[29548]: pgmap v1252: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:39 smithi070 ceph-mon[39923]: pgmap v1252: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:39 smithi155 ceph-mon[32688]: pgmap v1252: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:40.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:57:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:57:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:41 smithi070 ceph-mon[29548]: pgmap v1253: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:41 smithi070 ceph-mon[39923]: pgmap v1253: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:41 smithi155 ceph-mon[32688]: pgmap v1253: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:57:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:57:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:57:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:43 smithi070 ceph-mon[29548]: pgmap v1254: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:43 smithi070 ceph-mon[39923]: pgmap v1254: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:43 smithi155 ceph-mon[32688]: pgmap v1254: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:45 smithi070 ceph-mon[29548]: pgmap v1255: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:45 smithi070 ceph-mon[39923]: pgmap v1255: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:45 smithi155 ceph-mon[32688]: pgmap v1255: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:47 smithi070 ceph-mon[29548]: pgmap v1256: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:47 smithi070 ceph-mon[39923]: pgmap v1256: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:47 smithi155 ceph-mon[32688]: pgmap v1256: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:49 smithi070 ceph-mon[29548]: pgmap v1257: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:49 smithi070 ceph-mon[39923]: pgmap v1257: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:49 smithi155 ceph-mon[32688]: pgmap v1257: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:57:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:57:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:51 smithi070 ceph-mon[29548]: pgmap v1258: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:51 smithi070 ceph-mon[39923]: pgmap v1258: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:57:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:51 smithi155 ceph-mon[32688]: pgmap v1258: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:57:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:57:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:57:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:53 smithi070 ceph-mon[39923]: pgmap v1259: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:53 smithi070 ceph-mon[29548]: pgmap v1259: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:53 smithi155 ceph-mon[32688]: pgmap v1259: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:55 smithi070 ceph-mon[29548]: pgmap v1260: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:55 smithi070 ceph-mon[39923]: pgmap v1260: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:55 smithi155 ceph-mon[32688]: pgmap v1260: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:57:56.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.497 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.498 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.499 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.500 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.501 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.502 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.503 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.504 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.504 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.504 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.504 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.504 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.504 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.504 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:57:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:57 smithi070 ceph-mon[29548]: pgmap v1261: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:57 smithi070 ceph-mon[39923]: pgmap v1261: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:57 smithi155 ceph-mon[32688]: pgmap v1261: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:57:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:57:59 smithi070 ceph-mon[29548]: pgmap v1262: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:57:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:57:59 smithi070 ceph-mon[39923]: pgmap v1262: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:58:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:57:59 smithi155 ceph-mon[32688]: pgmap v1262: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:58:00.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:57:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:58:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:01 smithi070 ceph-mon[29548]: pgmap v1263: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:01 smithi070 ceph-mon[39923]: pgmap v1263: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:01 smithi155 ceph-mon[32688]: pgmap v1263: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:58:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:58:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:58:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:03 smithi070 ceph-mon[29548]: pgmap v1264: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:03 smithi070 ceph-mon[39923]: pgmap v1264: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:03 smithi155 ceph-mon[32688]: pgmap v1264: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:05 smithi070 ceph-mon[29548]: pgmap v1265: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:05 smithi070 ceph-mon[39923]: pgmap v1265: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:05 smithi155 ceph-mon[32688]: pgmap v1265: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:07 smithi070 ceph-mon[29548]: pgmap v1266: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:07 smithi070 ceph-mon[39923]: pgmap v1266: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:07 smithi155 ceph-mon[32688]: pgmap v1266: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:09 smithi070 ceph-mon[29548]: pgmap v1267: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:09 smithi070 ceph-mon[39923]: pgmap v1267: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:09 smithi155 ceph-mon[32688]: pgmap v1267: 292 pgs: 292 active+clean; 582 KiB data, 785 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:10.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:58:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:58:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:11 smithi070 ceph-mon[29548]: pgmap v1268: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:11 smithi070 ceph-mon[39923]: pgmap v1268: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:11 smithi155 ceph-mon[32688]: pgmap v1268: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:58:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:58:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:58:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:13 smithi070 ceph-mon[39923]: pgmap v1269: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:13.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:13 smithi070 ceph-mon[29548]: pgmap v1269: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:13 smithi155 ceph-mon[32688]: pgmap v1269: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:15 smithi070 ceph-mon[29548]: pgmap v1270: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:15 smithi070 ceph-mon[39923]: pgmap v1270: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:15 smithi155 ceph-mon[32688]: pgmap v1270: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:17 smithi070 ceph-mon[29548]: pgmap v1271: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:17 smithi070 ceph-mon[39923]: pgmap v1271: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:17 smithi155 ceph-mon[32688]: pgmap v1271: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:19 smithi070 ceph-mon[39923]: pgmap v1272: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:19.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:19 smithi070 ceph-mon[29548]: pgmap v1272: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:20.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:58:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:58:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:19 smithi155 ceph-mon[32688]: pgmap v1272: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:20 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:20.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:20 smithi070 ceph-mon[39923]: pgmap v1273: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:20 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:20.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:20 smithi070 ceph-mon[29548]: pgmap v1273: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:20 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:21.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:20 smithi155 ceph-mon[32688]: pgmap v1273: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T07:58:22.492 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:58:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:58:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:58:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:23 smithi070 ceph-mon[29548]: pgmap v1274: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:23 smithi070 ceph-mon[39923]: pgmap v1274: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:23 smithi155 ceph-mon[32688]: pgmap v1274: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:25 smithi070 ceph-mon[29548]: pgmap v1275: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:25 smithi070 ceph-mon[39923]: pgmap v1275: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:25 smithi155 ceph-mon[32688]: pgmap v1275: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:27 smithi070 ceph-mon[29548]: pgmap v1276: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:27 smithi070 ceph-mon[39923]: pgmap v1276: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:27 smithi155 ceph-mon[32688]: pgmap v1276: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:29 smithi070 ceph-mon[29548]: pgmap v1277: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:29 smithi070 ceph-mon[39923]: pgmap v1277: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:58:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:58:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:29 smithi155 ceph-mon[32688]: pgmap v1277: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:31 smithi070 ceph-mon[29548]: pgmap v1278: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:31 smithi070 ceph-mon[39923]: pgmap v1278: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:31 smithi155 ceph-mon[32688]: pgmap v1278: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:58:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:58:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:58:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:33 smithi070 ceph-mon[29548]: pgmap v1279: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:33 smithi070 ceph-mon[39923]: pgmap v1279: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:33 smithi155 ceph-mon[32688]: pgmap v1279: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:35 smithi070 ceph-mon[29548]: pgmap v1280: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:35 smithi070 ceph-mon[39923]: pgmap v1280: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:35 smithi155 ceph-mon[32688]: pgmap v1280: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:37 smithi070 ceph-mon[39923]: pgmap v1281: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:37 smithi070 ceph-mon[29548]: pgmap v1281: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:37 smithi155 ceph-mon[32688]: pgmap v1281: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:38.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:38 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:58:38.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:38 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:58:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:38 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:58:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[29548]: pgmap v1282: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:58:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:58:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:58:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[39923]: pgmap v1282: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:58:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:58:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:58:40.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:58:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:58:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:39 smithi155 ceph-mon[32688]: pgmap v1282: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:58:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:58:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:58:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:41 smithi070 ceph-mon[29548]: pgmap v1283: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:41 smithi070 ceph-mon[39923]: pgmap v1283: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:41 smithi155 ceph-mon[32688]: pgmap v1283: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:58:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:58:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:58:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:43 smithi070 ceph-mon[29548]: pgmap v1284: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:43 smithi070 ceph-mon[39923]: pgmap v1284: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:43 smithi155 ceph-mon[32688]: pgmap v1284: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:45 smithi070 ceph-mon[29548]: pgmap v1285: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:45.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:45 smithi070 ceph-mon[39923]: pgmap v1285: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:45 smithi155 ceph-mon[32688]: pgmap v1285: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:47 smithi070 ceph-mon[29548]: pgmap v1286: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:47 smithi070 ceph-mon[39923]: pgmap v1286: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:47 smithi155 ceph-mon[32688]: pgmap v1286: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:49 smithi070 ceph-mon[29548]: pgmap v1287: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:49 smithi070 ceph-mon[39923]: pgmap v1287: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:58:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:58:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:49 smithi155 ceph-mon[32688]: pgmap v1287: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:51.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:51 smithi070 ceph-mon[39923]: pgmap v1288: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:51 smithi070 ceph-mon[29548]: pgmap v1288: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:58:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:51 smithi155 ceph-mon[32688]: pgmap v1288: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:58:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:58:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:58:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:53 smithi070 ceph-mon[39923]: pgmap v1289: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:53 smithi070 ceph-mon[29548]: pgmap v1289: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:53 smithi155 ceph-mon[32688]: pgmap v1289: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:55 smithi070 ceph-mon[29548]: pgmap v1290: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:55 smithi070 ceph-mon[39923]: pgmap v1290: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:55 smithi155 ceph-mon[32688]: pgmap v1290: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:58:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:56.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:58:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:57 smithi070 ceph-mon[29548]: pgmap v1291: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:57 smithi070 ceph-mon[39923]: pgmap v1291: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:57 smithi155 ceph-mon[32688]: pgmap v1291: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:58:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:58:59 smithi070 ceph-mon[29548]: pgmap v1292: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:58:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:58:59 smithi070 ceph-mon[39923]: pgmap v1292: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:00.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:58:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:59:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:58:59 smithi155 ceph-mon[32688]: pgmap v1292: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:01 smithi070 ceph-mon[29548]: pgmap v1293: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:01 smithi070 ceph-mon[39923]: pgmap v1293: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:01 smithi155 ceph-mon[32688]: pgmap v1293: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:59:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:59:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:59:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:03 smithi070 ceph-mon[29548]: pgmap v1294: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:03 smithi070 ceph-mon[39923]: pgmap v1294: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:03 smithi155 ceph-mon[32688]: pgmap v1294: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:05 smithi070 ceph-mon[29548]: pgmap v1295: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:05 smithi070 ceph-mon[39923]: pgmap v1295: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:05 smithi155 ceph-mon[32688]: pgmap v1295: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:07 smithi070 ceph-mon[29548]: pgmap v1296: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:07 smithi070 ceph-mon[39923]: pgmap v1296: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:07 smithi155 ceph-mon[32688]: pgmap v1296: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:09 smithi070 ceph-mon[29548]: pgmap v1297: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:09 smithi070 ceph-mon[39923]: pgmap v1297: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:10.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:59:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:59:10.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:09 smithi155 ceph-mon[32688]: pgmap v1297: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:11 smithi070 ceph-mon[29548]: pgmap v1298: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:11 smithi070 ceph-mon[39923]: pgmap v1298: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:11 smithi155 ceph-mon[32688]: pgmap v1298: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:59:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:59:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:59:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:13 smithi070 ceph-mon[29548]: pgmap v1299: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:13 smithi070 ceph-mon[39923]: pgmap v1299: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:13 smithi155 ceph-mon[32688]: pgmap v1299: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:15 smithi070 ceph-mon[29548]: pgmap v1300: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:15 smithi070 ceph-mon[39923]: pgmap v1300: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:15 smithi155 ceph-mon[32688]: pgmap v1300: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:17 smithi070 ceph-mon[29548]: pgmap v1301: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:17 smithi070 ceph-mon[39923]: pgmap v1301: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:17 smithi155 ceph-mon[32688]: pgmap v1301: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:19 smithi070 ceph-mon[29548]: pgmap v1302: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:19 smithi070 ceph-mon[39923]: pgmap v1302: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:19 smithi155 ceph-mon[32688]: pgmap v1302: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:20.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:59:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:59:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:21 smithi070 ceph-mon[29548]: pgmap v1303: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:21 smithi070 ceph-mon[39923]: pgmap v1303: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:21 smithi155 ceph-mon[32688]: pgmap v1303: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:59:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:59:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:59:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:23 smithi070 ceph-mon[29548]: pgmap v1304: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:23 smithi070 ceph-mon[39923]: pgmap v1304: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:23 smithi155 ceph-mon[32688]: pgmap v1304: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:25 smithi070 ceph-mon[29548]: pgmap v1305: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:25 smithi070 ceph-mon[39923]: pgmap v1305: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:25 smithi155 ceph-mon[32688]: pgmap v1305: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:27 smithi070 ceph-mon[29548]: pgmap v1306: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:27 smithi070 ceph-mon[39923]: pgmap v1306: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:27 smithi155 ceph-mon[32688]: pgmap v1306: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:29 smithi070 ceph-mon[29548]: pgmap v1307: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:29 smithi070 ceph-mon[39923]: pgmap v1307: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:30.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:59:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:59:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:29 smithi155 ceph-mon[32688]: pgmap v1307: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:31 smithi070 ceph-mon[29548]: pgmap v1308: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:31 smithi070 ceph-mon[39923]: pgmap v1308: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:31 smithi155 ceph-mon[32688]: pgmap v1308: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:59:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:59:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:59:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:33 smithi070 ceph-mon[29548]: pgmap v1309: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:33 smithi070 ceph-mon[39923]: pgmap v1309: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:33 smithi155 ceph-mon[32688]: pgmap v1309: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:35 smithi070 ceph-mon[29548]: pgmap v1310: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:35 smithi070 ceph-mon[39923]: pgmap v1310: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:35 smithi155 ceph-mon[32688]: pgmap v1310: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.509 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.510 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.511 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.512 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.513 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.514 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.515 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.516 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T07:59:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:37 smithi070 ceph-mon[29548]: pgmap v1311: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:37 smithi070 ceph-mon[39923]: pgmap v1311: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:37 smithi155 ceph-mon[32688]: pgmap v1311: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[29548]: pgmap v1312: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:59:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:59:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:59:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:59:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[39923]: pgmap v1312: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:59:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:59:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:59:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:39 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:59:40.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:59:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:59:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:39 smithi155 ceph-mon[32688]: pgmap v1312: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:59:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:59:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:59:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:39 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T07:59:41.505 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 07:59:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=cleanup t=2024-02-10T07:59:41.015335095Z level=info msg="Completed cleanup jobs" duration=2.962401ms 2024-02-10T07:59:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:41 smithi070 ceph-mon[29548]: pgmap v1313: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:41 smithi070 ceph-mon[39923]: pgmap v1313: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:41 smithi155 ceph-mon[32688]: pgmap v1313: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:59:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:59:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:59:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:43 smithi070 ceph-mon[29548]: pgmap v1314: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:43 smithi070 ceph-mon[39923]: pgmap v1314: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:43 smithi155 ceph-mon[32688]: pgmap v1314: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T07:59:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:45 smithi070 ceph-mon[29548]: pgmap v1315: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:45 smithi070 ceph-mon[39923]: pgmap v1315: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:45 smithi155 ceph-mon[32688]: pgmap v1315: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:47 smithi070 ceph-mon[29548]: pgmap v1316: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:47 smithi070 ceph-mon[39923]: pgmap v1316: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:47 smithi155 ceph-mon[32688]: pgmap v1316: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:49 smithi070 ceph-mon[29548]: pgmap v1317: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:49 smithi070 ceph-mon[39923]: pgmap v1317: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:50.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:59:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T07:59:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:49 smithi155 ceph-mon[32688]: pgmap v1317: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T07:59:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:51 smithi070 ceph-mon[29548]: pgmap v1318: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:51 smithi070 ceph-mon[39923]: pgmap v1318: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T07:59:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:51 smithi155 ceph-mon[32688]: pgmap v1318: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 07:59:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:07:59:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:59:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:53 smithi070 ceph-mon[39923]: pgmap v1319: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:53 smithi070 ceph-mon[29548]: pgmap v1319: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:53 smithi155 ceph-mon[32688]: pgmap v1319: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:55.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:54 smithi155 ceph-mon[32688]: pgmap v1320: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:54 smithi070 ceph-mon[29548]: pgmap v1320: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:54 smithi070 ceph-mon[39923]: pgmap v1320: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T07:59:56.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:56.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:56.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:59:57.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:57 smithi070 ceph-mon[29548]: pgmap v1321: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:57.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:57 smithi070 ceph-mon[39923]: pgmap v1321: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:57.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:57 smithi155 ceph-mon[32688]: pgmap v1321: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:59.893 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 07:59:59 smithi155 ceph-mon[32688]: pgmap v1322: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 07:59:59 smithi070 ceph-mon[29548]: pgmap v1322: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T07:59:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 07:59:59 smithi070 ceph-mon[39923]: pgmap v1322: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:00.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 07:59:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:00:00.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:00 smithi070 ceph-mon[29548]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T08:00:00.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:00 smithi070 ceph-mon[39923]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T08:00:01.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:00 smithi155 ceph-mon[32688]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T08:00:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:01 smithi070 ceph-mon[29548]: pgmap v1323: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:01 smithi070 ceph-mon[39923]: pgmap v1323: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:01 smithi155 ceph-mon[32688]: pgmap v1323: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:00:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:00:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:00:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:03 smithi070 ceph-mon[29548]: pgmap v1324: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:03 smithi070 ceph-mon[39923]: pgmap v1324: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:03 smithi155 ceph-mon[32688]: pgmap v1324: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:05 smithi070 ceph-mon[29548]: pgmap v1325: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:05 smithi070 ceph-mon[39923]: pgmap v1325: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:05 smithi155 ceph-mon[32688]: pgmap v1325: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:07 smithi070 ceph-mon[29548]: pgmap v1326: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:07 smithi070 ceph-mon[39923]: pgmap v1326: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:07 smithi155 ceph-mon[32688]: pgmap v1326: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:09.904 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:09 smithi155 ceph-mon[32688]: pgmap v1327: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:09 smithi070 ceph-mon[39923]: pgmap v1327: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:09 smithi070 ceph-mon[29548]: pgmap v1327: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:10.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:00:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:00:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:11 smithi070 ceph-mon[29548]: pgmap v1328: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:11 smithi070 ceph-mon[39923]: pgmap v1328: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:11 smithi155 ceph-mon[32688]: pgmap v1328: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:00:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:00:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:00:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:13 smithi070 ceph-mon[29548]: pgmap v1329: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:13 smithi070 ceph-mon[39923]: pgmap v1329: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:13 smithi155 ceph-mon[32688]: pgmap v1329: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:15 smithi070 ceph-mon[29548]: pgmap v1330: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:15.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:15 smithi070 ceph-mon[39923]: pgmap v1330: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:15 smithi155 ceph-mon[32688]: pgmap v1330: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:17 smithi070 ceph-mon[29548]: pgmap v1331: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:17 smithi070 ceph-mon[39923]: pgmap v1331: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:17 smithi155 ceph-mon[32688]: pgmap v1331: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:19.905 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:19 smithi155 ceph-mon[32688]: pgmap v1332: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:19 smithi070 ceph-mon[39923]: pgmap v1332: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:19 smithi070 ceph-mon[29548]: pgmap v1332: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:20.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:00:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:00:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:21 smithi070 ceph-mon[39923]: pgmap v1333: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:21 smithi070 ceph-mon[29548]: pgmap v1333: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:21 smithi155 ceph-mon[32688]: pgmap v1333: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:00:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:00:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:00:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:23 smithi070 ceph-mon[39923]: pgmap v1334: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:23 smithi070 ceph-mon[29548]: pgmap v1334: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:23 smithi155 ceph-mon[32688]: pgmap v1334: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:25 smithi070 ceph-mon[29548]: pgmap v1335: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:25 smithi070 ceph-mon[39923]: pgmap v1335: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:25 smithi155 ceph-mon[32688]: pgmap v1335: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:27.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:27 smithi070 ceph-mon[39923]: pgmap v1336: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:27 smithi070 ceph-mon[29548]: pgmap v1336: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:27 smithi155 ceph-mon[32688]: pgmap v1336: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:29.910 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:29 smithi155 ceph-mon[32688]: pgmap v1337: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:00:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:29 smithi070 ceph-mon[39923]: pgmap v1337: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:00:29.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:29 smithi070 ceph-mon[29548]: pgmap v1337: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:00:30.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:00:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:00:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:31 smithi070 ceph-mon[29548]: pgmap v1338: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:31 smithi070 ceph-mon[39923]: pgmap v1338: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:31 smithi155 ceph-mon[32688]: pgmap v1338: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:00:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:00:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:00:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:33 smithi070 ceph-mon[29548]: pgmap v1339: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:33 smithi070 ceph-mon[39923]: pgmap v1339: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:33 smithi155 ceph-mon[32688]: pgmap v1339: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:35 smithi070 ceph-mon[29548]: pgmap v1340: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:35 smithi070 ceph-mon[39923]: pgmap v1340: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:35 smithi155 ceph-mon[32688]: pgmap v1340: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:37 smithi070 ceph-mon[29548]: pgmap v1341: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:37 smithi070 ceph-mon[39923]: pgmap v1341: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:37 smithi155 ceph-mon[32688]: pgmap v1341: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:39.915 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:39 smithi155 ceph-mon[32688]: pgmap v1342: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:00:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:39 smithi070 ceph-mon[29548]: pgmap v1342: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:00:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:39 smithi070 ceph-mon[39923]: pgmap v1342: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:00:40.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:00:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:00:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:00:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:00:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:00:40.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:00:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:00:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:00:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:00:40.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:00:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:00:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:00:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:00:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:00:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:41 smithi070 ceph-mon[29548]: pgmap v1343: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:41 smithi070 ceph-mon[39923]: pgmap v1343: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:41 smithi155 ceph-mon[32688]: pgmap v1343: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:00:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:00:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:00:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:43 smithi070 ceph-mon[29548]: pgmap v1344: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:43 smithi070 ceph-mon[39923]: pgmap v1344: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:43 smithi155 ceph-mon[32688]: pgmap v1344: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:45 smithi070 ceph-mon[29548]: pgmap v1345: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:45 smithi070 ceph-mon[39923]: pgmap v1345: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:45 smithi155 ceph-mon[32688]: pgmap v1345: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:47 smithi070 ceph-mon[29548]: pgmap v1346: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:47 smithi070 ceph-mon[39923]: pgmap v1346: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:47 smithi155 ceph-mon[32688]: pgmap v1346: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:49.921 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:49 smithi155 ceph-mon[32688]: pgmap v1347: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:49 smithi070 ceph-mon[29548]: pgmap v1347: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:49 smithi070 ceph-mon[39923]: pgmap v1347: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:50.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:00:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:00:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:51 smithi070 ceph-mon[29548]: pgmap v1348: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:51 smithi070 ceph-mon[39923]: pgmap v1348: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:00:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:51 smithi155 ceph-mon[32688]: pgmap v1348: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:00:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:00:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:00:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:53 smithi070 ceph-mon[39923]: pgmap v1349: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:53 smithi070 ceph-mon[29548]: pgmap v1349: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:53 smithi155 ceph-mon[32688]: pgmap v1349: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:55 smithi070 ceph-mon[29548]: pgmap v1350: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:55 smithi070 ceph-mon[39923]: pgmap v1350: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:55 smithi155 ceph-mon[32688]: pgmap v1350: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:00:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:56.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:00:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:57 smithi070 ceph-mon[29548]: pgmap v1351: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:57 smithi070 ceph-mon[39923]: pgmap v1351: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:57 smithi155 ceph-mon[32688]: pgmap v1351: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:59.932 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:00:59 smithi155 ceph-mon[32688]: pgmap v1352: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:00:59 smithi070 ceph-mon[29548]: pgmap v1352: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:00:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:00:59 smithi070 ceph-mon[39923]: pgmap v1352: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:00.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:00:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:01:01.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:01.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:01 smithi155 ceph-mon[32688]: pgmap v1353: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:01 smithi070 ceph-mon[29548]: pgmap v1353: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:01 smithi070 ceph-mon[39923]: pgmap v1353: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:01:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:01:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:01:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:03 smithi070 ceph-mon[29548]: pgmap v1354: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:03 smithi070 ceph-mon[39923]: pgmap v1354: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:03 smithi155 ceph-mon[32688]: pgmap v1354: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:05 smithi070 ceph-mon[29548]: pgmap v1355: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:05 smithi070 ceph-mon[39923]: pgmap v1355: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:05 smithi155 ceph-mon[32688]: pgmap v1355: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:07 smithi070 ceph-mon[29548]: pgmap v1356: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:07 smithi070 ceph-mon[39923]: pgmap v1356: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:07 smithi155 ceph-mon[32688]: pgmap v1356: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:09.940 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:09 smithi155 ceph-mon[32688]: pgmap v1357: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:09 smithi070 ceph-mon[29548]: pgmap v1357: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:09 smithi070 ceph-mon[39923]: pgmap v1357: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:10.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:01:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:01:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:11 smithi070 ceph-mon[29548]: pgmap v1358: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:11 smithi070 ceph-mon[39923]: pgmap v1358: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:11 smithi155 ceph-mon[32688]: pgmap v1358: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:01:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:01:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:01:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:13 smithi070 ceph-mon[29548]: pgmap v1359: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:13 smithi070 ceph-mon[39923]: pgmap v1359: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:13 smithi155 ceph-mon[32688]: pgmap v1359: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:15 smithi070 ceph-mon[29548]: pgmap v1360: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:15 smithi070 ceph-mon[39923]: pgmap v1360: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:15 smithi155 ceph-mon[32688]: pgmap v1360: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:17.520 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.521 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.522 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.523 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.524 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.525 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.526 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.527 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.527 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.527 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.527 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.527 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:01:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:17 smithi070 ceph-mon[29548]: pgmap v1361: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:17 smithi070 ceph-mon[39923]: pgmap v1361: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:17 smithi155 ceph-mon[32688]: pgmap v1361: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:19.951 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:19 smithi155 ceph-mon[32688]: pgmap v1362: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:19 smithi070 ceph-mon[29548]: pgmap v1362: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:19 smithi070 ceph-mon[39923]: pgmap v1362: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:20.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:01:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:01:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:21 smithi070 ceph-mon[29548]: pgmap v1363: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:21 smithi070 ceph-mon[39923]: pgmap v1363: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:21 smithi155 ceph-mon[32688]: pgmap v1363: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:01:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:01:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:01:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:23 smithi070 ceph-mon[29548]: pgmap v1364: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:23 smithi070 ceph-mon[39923]: pgmap v1364: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:23 smithi155 ceph-mon[32688]: pgmap v1364: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:25 smithi070 ceph-mon[29548]: pgmap v1365: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:25 smithi070 ceph-mon[39923]: pgmap v1365: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:25 smithi155 ceph-mon[32688]: pgmap v1365: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:27 smithi070 ceph-mon[29548]: pgmap v1366: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:27 smithi070 ceph-mon[39923]: pgmap v1366: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:27 smithi155 ceph-mon[32688]: pgmap v1366: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:28.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:28 smithi070 ceph-mon[29548]: pgmap v1367: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:28.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:28 smithi070 ceph-mon[39923]: pgmap v1367: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:29.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:28 smithi155 ceph-mon[32688]: pgmap v1367: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:30.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:01:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:01:31.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:31 smithi070 ceph-mon[29548]: pgmap v1368: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:31 smithi070 ceph-mon[39923]: pgmap v1368: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:31 smithi155 ceph-mon[32688]: pgmap v1368: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:01:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:01:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:01:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:33 smithi070 ceph-mon[29548]: pgmap v1369: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:33 smithi070 ceph-mon[39923]: pgmap v1369: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:33 smithi155 ceph-mon[32688]: pgmap v1369: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:35 smithi070 ceph-mon[29548]: pgmap v1370: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:35 smithi070 ceph-mon[39923]: pgmap v1370: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:35 smithi155 ceph-mon[32688]: pgmap v1370: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:37 smithi070 ceph-mon[29548]: pgmap v1371: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:37 smithi070 ceph-mon[39923]: pgmap v1371: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:37 smithi155 ceph-mon[32688]: pgmap v1371: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:39.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:39 smithi155 ceph-mon[32688]: pgmap v1372: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:39 smithi070 ceph-mon[29548]: pgmap v1372: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:39.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:39 smithi070 ceph-mon[39923]: pgmap v1372: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:40.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:01:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:01:40.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:40 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:01:40.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:40 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:01:41.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:40 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:01:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:41 smithi070 ceph-mon[39923]: pgmap v1373: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:41 smithi070 ceph-mon[29548]: pgmap v1373: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:42.003 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:42.003 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:41 smithi155 ceph-mon[32688]: pgmap v1373: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:42.003 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:01:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:01:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:01:43.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:43 smithi155 ceph-mon[32688]: pgmap v1374: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:43.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:43 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:43.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:43 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:43 smithi070 ceph-mon[29548]: pgmap v1374: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:43 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:43 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:43 smithi070 ceph-mon[39923]: pgmap v1374: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:43 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:43 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[29548]: pgmap v1375: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[39923]: pgmap v1375: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:01:45.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:45 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:45 smithi155 ceph-mon[32688]: pgmap v1375: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:45 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:45 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:45 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:01:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:45 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:01:45.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:45 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:01:47.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:47 smithi070 ceph-mon[29548]: pgmap v1376: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:47.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:47 smithi070 ceph-mon[39923]: pgmap v1376: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:47.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:47 smithi155 ceph-mon[32688]: pgmap v1376: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:49.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:49 smithi155 ceph-mon[32688]: pgmap v1377: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:49 smithi070 ceph-mon[29548]: pgmap v1377: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:49 smithi070 ceph-mon[39923]: pgmap v1377: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:50.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:01:49 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:01:51.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:51.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:51 smithi070 ceph-mon[39923]: pgmap v1378: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:51 smithi070 ceph-mon[29548]: pgmap v1378: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:01:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:51 smithi155 ceph-mon[32688]: pgmap v1378: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:01:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:01:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:01:53.767 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:53 smithi070 ceph-mon[39923]: pgmap v1379: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:53.767 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:53 smithi070 ceph-mon[29548]: pgmap v1379: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:53 smithi155 ceph-mon[32688]: pgmap v1379: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:55 smithi070 ceph-mon[39923]: pgmap v1380: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:55 smithi070 ceph-mon[29548]: pgmap v1380: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:55 smithi155 ceph-mon[32688]: pgmap v1380: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:01:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:56.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:01:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:57 smithi070 ceph-mon[29548]: pgmap v1381: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:57 smithi070 ceph-mon[39923]: pgmap v1381: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:57 smithi155 ceph-mon[32688]: pgmap v1381: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:01:59.985 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:01:59 smithi155 ceph-mon[32688]: pgmap v1382: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:01:59 smithi070 ceph-mon[29548]: pgmap v1382: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:01:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:01:59 smithi070 ceph-mon[39923]: pgmap v1382: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:02:00.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:01:59 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:02:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:01 smithi070 ceph-mon[29548]: pgmap v1383: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:01 smithi070 ceph-mon[39923]: pgmap v1383: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:01 smithi155 ceph-mon[32688]: pgmap v1383: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:02:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:02:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:02:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:03 smithi070 ceph-mon[29548]: pgmap v1384: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:03 smithi070 ceph-mon[39923]: pgmap v1384: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:03 smithi155 ceph-mon[32688]: pgmap v1384: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:05 smithi070 ceph-mon[29548]: pgmap v1385: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:05 smithi070 ceph-mon[39923]: pgmap v1385: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:05 smithi155 ceph-mon[32688]: pgmap v1385: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:07 smithi070 ceph-mon[29548]: pgmap v1386: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:07 smithi070 ceph-mon[39923]: pgmap v1386: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:07 smithi155 ceph-mon[32688]: pgmap v1386: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:09 smithi070 ceph-mon[29548]: pgmap v1387: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:09 smithi070 ceph-mon[39923]: pgmap v1387: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:09.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:09 smithi155 ceph-mon[32688]: pgmap v1387: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:10.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:02:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:02:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:11 smithi070 ceph-mon[29548]: pgmap v1388: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:11 smithi070 ceph-mon[39923]: pgmap v1388: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:11 smithi155 ceph-mon[32688]: pgmap v1388: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:02:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:02:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:02:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:13 smithi070 ceph-mon[29548]: pgmap v1389: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:13 smithi070 ceph-mon[39923]: pgmap v1389: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:13 smithi155 ceph-mon[32688]: pgmap v1389: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:15 smithi070 ceph-mon[29548]: pgmap v1390: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:15 smithi070 ceph-mon[39923]: pgmap v1390: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:15 smithi155 ceph-mon[32688]: pgmap v1390: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:17 smithi070 ceph-mon[29548]: pgmap v1391: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:17 smithi070 ceph-mon[39923]: pgmap v1391: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:17 smithi155 ceph-mon[32688]: pgmap v1391: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:19 smithi070 ceph-mon[29548]: pgmap v1392: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:19 smithi070 ceph-mon[39923]: pgmap v1392: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:19 smithi155 ceph-mon[32688]: pgmap v1392: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:20.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:02:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:02:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:21 smithi070 ceph-mon[29548]: pgmap v1393: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:21 smithi070 ceph-mon[39923]: pgmap v1393: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:21 smithi155 ceph-mon[32688]: pgmap v1393: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:02:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:02:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:02:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:23 smithi070 ceph-mon[39923]: pgmap v1394: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:23 smithi070 ceph-mon[29548]: pgmap v1394: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:23 smithi155 ceph-mon[32688]: pgmap v1394: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:25 smithi070 ceph-mon[29548]: pgmap v1395: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:25 smithi070 ceph-mon[39923]: pgmap v1395: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:25 smithi155 ceph-mon[32688]: pgmap v1395: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:27 smithi070 ceph-mon[29548]: pgmap v1396: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:27 smithi070 ceph-mon[39923]: pgmap v1396: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:27 smithi155 ceph-mon[32688]: pgmap v1396: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:29 smithi070 ceph-mon[29548]: pgmap v1397: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:02:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:29 smithi070 ceph-mon[39923]: pgmap v1397: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:02:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:29 smithi155 ceph-mon[32688]: pgmap v1397: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:02:30.507 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:02:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:02:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:31 smithi070 ceph-mon[29548]: pgmap v1398: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:31 smithi070 ceph-mon[39923]: pgmap v1398: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:31 smithi155 ceph-mon[32688]: pgmap v1398: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:02:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:02:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:02:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:33 smithi070 ceph-mon[29548]: pgmap v1399: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:33 smithi070 ceph-mon[39923]: pgmap v1399: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:33 smithi155 ceph-mon[32688]: pgmap v1399: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:35 smithi070 ceph-mon[29548]: pgmap v1400: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:35 smithi070 ceph-mon[39923]: pgmap v1400: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:35 smithi155 ceph-mon[32688]: pgmap v1400: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:37 smithi070 ceph-mon[29548]: pgmap v1401: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:37 smithi070 ceph-mon[39923]: pgmap v1401: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:37 smithi155 ceph-mon[32688]: pgmap v1401: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:39 smithi070 ceph-mon[29548]: pgmap v1402: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:39 smithi070 ceph-mon[39923]: pgmap v1402: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:39 smithi155 ceph-mon[32688]: pgmap v1402: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:40.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:02:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:02:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:41 smithi070 ceph-mon[29548]: pgmap v1403: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:41 smithi070 ceph-mon[39923]: pgmap v1403: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:41 smithi155 ceph-mon[32688]: pgmap v1403: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:02:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:02:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:02:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:43 smithi070 ceph-mon[39923]: pgmap v1404: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:43 smithi070 ceph-mon[29548]: pgmap v1404: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:43 smithi155 ceph-mon[32688]: pgmap v1404: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:45 smithi070 ceph-mon[29548]: pgmap v1405: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:45 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:02:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:45 smithi070 ceph-mon[39923]: pgmap v1405: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:45 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:02:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:45 smithi155 ceph-mon[32688]: pgmap v1405: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:45 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:02:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:46 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:02:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:46 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:02:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:46 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:02:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:46 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:02:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:46 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:02:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:46 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:02:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:46 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:02:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:46 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:02:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:46 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:02:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:47 smithi070 ceph-mon[29548]: pgmap v1406: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:47 smithi070 ceph-mon[39923]: pgmap v1406: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:47 smithi155 ceph-mon[32688]: pgmap v1406: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:49 smithi070 ceph-mon[29548]: pgmap v1407: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:49 smithi070 ceph-mon[39923]: pgmap v1407: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:49 smithi155 ceph-mon[32688]: pgmap v1407: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:50.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:02:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:02:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:51 smithi070 ceph-mon[29548]: pgmap v1408: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:51.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:51 smithi070 ceph-mon[39923]: pgmap v1408: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:02:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:51 smithi155 ceph-mon[32688]: pgmap v1408: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:02:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:02:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:02:53.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:53 smithi070 ceph-mon[39923]: pgmap v1409: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:53.772 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:53 smithi070 ceph-mon[29548]: pgmap v1409: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:53 smithi155 ceph-mon[32688]: pgmap v1409: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:55 smithi070 ceph-mon[29548]: pgmap v1410: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:55 smithi070 ceph-mon[39923]: pgmap v1410: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:55 smithi155 ceph-mon[32688]: pgmap v1410: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:02:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:02:57.532 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.532 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.533 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.534 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.535 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.536 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.537 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.538 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.539 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:02:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:57 smithi070 ceph-mon[39923]: pgmap v1411: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:57 smithi070 ceph-mon[29548]: pgmap v1411: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:57 smithi155 ceph-mon[32688]: pgmap v1411: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:02:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:02:59 smithi070 ceph-mon[29548]: pgmap v1412: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:02:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:02:59 smithi070 ceph-mon[39923]: pgmap v1412: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:03:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:02:59 smithi155 ceph-mon[32688]: pgmap v1412: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:03:00.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:03:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:03:01.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:01 smithi070 ceph-mon[39923]: pgmap v1413: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:01 smithi070 ceph-mon[29548]: pgmap v1413: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:01 smithi155 ceph-mon[32688]: pgmap v1413: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:03:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:03:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:03:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:03 smithi070 ceph-mon[39923]: pgmap v1414: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:03 smithi070 ceph-mon[29548]: pgmap v1414: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:03 smithi155 ceph-mon[32688]: pgmap v1414: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:05 smithi070 ceph-mon[29548]: pgmap v1415: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:05 smithi070 ceph-mon[39923]: pgmap v1415: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:05 smithi155 ceph-mon[32688]: pgmap v1415: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:07 smithi070 ceph-mon[29548]: pgmap v1416: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:07 smithi070 ceph-mon[39923]: pgmap v1416: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:07 smithi155 ceph-mon[32688]: pgmap v1416: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:09 smithi070 ceph-mon[39923]: pgmap v1417: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:09 smithi070 ceph-mon[29548]: pgmap v1417: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:09 smithi155 ceph-mon[32688]: pgmap v1417: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:10.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:03:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:03:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:11 smithi070 ceph-mon[29548]: pgmap v1418: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:11 smithi070 ceph-mon[39923]: pgmap v1418: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:11 smithi155 ceph-mon[32688]: pgmap v1418: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:03:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:03:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:03:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:13 smithi070 ceph-mon[29548]: pgmap v1419: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:13 smithi070 ceph-mon[39923]: pgmap v1419: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:13 smithi155 ceph-mon[32688]: pgmap v1419: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:15 smithi070 ceph-mon[29548]: pgmap v1420: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:15 smithi070 ceph-mon[39923]: pgmap v1420: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:15 smithi155 ceph-mon[32688]: pgmap v1420: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:17 smithi070 ceph-mon[29548]: pgmap v1421: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:17 smithi070 ceph-mon[39923]: pgmap v1421: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:17 smithi155 ceph-mon[32688]: pgmap v1421: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:19 smithi070 ceph-mon[29548]: pgmap v1422: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:19 smithi070 ceph-mon[39923]: pgmap v1422: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:19 smithi155 ceph-mon[32688]: pgmap v1422: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:20.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:03:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:03:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:21 smithi070 ceph-mon[39923]: pgmap v1423: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:21 smithi070 ceph-mon[29548]: pgmap v1423: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:21 smithi155 ceph-mon[32688]: pgmap v1423: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:03:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:03:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:03:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:23 smithi070 ceph-mon[29548]: pgmap v1424: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:23 smithi070 ceph-mon[39923]: pgmap v1424: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:23 smithi155 ceph-mon[32688]: pgmap v1424: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:25 smithi070 ceph-mon[29548]: pgmap v1425: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:25.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:25 smithi070 ceph-mon[39923]: pgmap v1425: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:25 smithi155 ceph-mon[32688]: pgmap v1425: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:27 smithi070 ceph-mon[29548]: pgmap v1426: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:27 smithi070 ceph-mon[39923]: pgmap v1426: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:27 smithi155 ceph-mon[32688]: pgmap v1426: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:29 smithi070 ceph-mon[29548]: pgmap v1427: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:29 smithi070 ceph-mon[39923]: pgmap v1427: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:29 smithi155 ceph-mon[32688]: pgmap v1427: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:30.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:03:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:03:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:31 smithi070 ceph-mon[39923]: pgmap v1428: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:31 smithi070 ceph-mon[29548]: pgmap v1428: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:31 smithi155 ceph-mon[32688]: pgmap v1428: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:03:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:03:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:03:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:33 smithi070 ceph-mon[39923]: pgmap v1429: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:33 smithi070 ceph-mon[29548]: pgmap v1429: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:33 smithi155 ceph-mon[32688]: pgmap v1429: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:35.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:35 smithi070 ceph-mon[39923]: pgmap v1430: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:35 smithi070 ceph-mon[29548]: pgmap v1430: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:35 smithi155 ceph-mon[32688]: pgmap v1430: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:37.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:37 smithi070 ceph-mon[39923]: pgmap v1431: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:37.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:37 smithi070 ceph-mon[29548]: pgmap v1431: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:37 smithi155 ceph-mon[32688]: pgmap v1431: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:39 smithi070 ceph-mon[29548]: pgmap v1432: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:39 smithi070 ceph-mon[39923]: pgmap v1432: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:39 smithi155 ceph-mon[32688]: pgmap v1432: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:40.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:03:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:03:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:41.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:41 smithi070 ceph-mon[39923]: pgmap v1433: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:41 smithi070 ceph-mon[29548]: pgmap v1433: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:41 smithi155 ceph-mon[32688]: pgmap v1433: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:03:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:03:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:03:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:43 smithi070 ceph-mon[39923]: pgmap v1434: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:43 smithi070 ceph-mon[29548]: pgmap v1434: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:43 smithi155 ceph-mon[32688]: pgmap v1434: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:45 smithi070 ceph-mon[29548]: pgmap v1435: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:45 smithi070 ceph-mon[39923]: pgmap v1435: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:45 smithi155 ceph-mon[32688]: pgmap v1435: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:46 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:03:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:46 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:46.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:46 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:46.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:46 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:03:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:46 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:46.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:46 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:46 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:03:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:46 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:47.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:46 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[29548]: pgmap v1436: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:03:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:03:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[39923]: pgmap v1436: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:03:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:03:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:47 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:47 smithi155 ceph-mon[32688]: pgmap v1436: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:47 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:03:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:47 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:03:48.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:47 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:03:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:49 smithi070 ceph-mon[29548]: pgmap v1437: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:49 smithi070 ceph-mon[39923]: pgmap v1437: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:49 smithi155 ceph-mon[32688]: pgmap v1437: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:50.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:03:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:03:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:51 smithi070 ceph-mon[29548]: pgmap v1438: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:51 smithi070 ceph-mon[39923]: pgmap v1438: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:03:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:51 smithi155 ceph-mon[32688]: pgmap v1438: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:03:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:03:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:03:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:53 smithi070 ceph-mon[39923]: pgmap v1439: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:53 smithi070 ceph-mon[29548]: pgmap v1439: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:53 smithi155 ceph-mon[32688]: pgmap v1439: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:55 smithi070 ceph-mon[29548]: pgmap v1440: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:55 smithi070 ceph-mon[39923]: pgmap v1440: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:55 smithi155 ceph-mon[32688]: pgmap v1440: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:03:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:03:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:57 smithi070 ceph-mon[39923]: pgmap v1441: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:57 smithi070 ceph-mon[29548]: pgmap v1441: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:57 smithi155 ceph-mon[32688]: pgmap v1441: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:03:59 smithi070 ceph-mon[29548]: pgmap v1442: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:03:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:03:59 smithi070 ceph-mon[39923]: pgmap v1442: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:03:59 smithi155 ceph-mon[32688]: pgmap v1442: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:00.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:04:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:04:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:01 smithi070 ceph-mon[29548]: pgmap v1443: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:01 smithi070 ceph-mon[39923]: pgmap v1443: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:02.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:02.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:01 smithi155 ceph-mon[32688]: pgmap v1443: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:04:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:04:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:04:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:03 smithi070 ceph-mon[29548]: pgmap v1444: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:03 smithi070 ceph-mon[39923]: pgmap v1444: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:03 smithi155 ceph-mon[32688]: pgmap v1444: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:05 smithi070 ceph-mon[39923]: pgmap v1445: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:05 smithi070 ceph-mon[29548]: pgmap v1445: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:05 smithi155 ceph-mon[32688]: pgmap v1445: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:07 smithi070 ceph-mon[39923]: pgmap v1446: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:07 smithi070 ceph-mon[29548]: pgmap v1446: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:07 smithi155 ceph-mon[32688]: pgmap v1446: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:09 smithi070 ceph-mon[29548]: pgmap v1447: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:09 smithi070 ceph-mon[39923]: pgmap v1447: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:09 smithi155 ceph-mon[32688]: pgmap v1447: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:10.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:04:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:04:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:11 smithi070 ceph-mon[29548]: pgmap v1448: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:11.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:11 smithi070 ceph-mon[39923]: pgmap v1448: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:11.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:11 smithi155 ceph-mon[32688]: pgmap v1448: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:04:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:04:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:04:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:13 smithi070 ceph-mon[29548]: pgmap v1449: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:13 smithi070 ceph-mon[39923]: pgmap v1449: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:13 smithi155 ceph-mon[32688]: pgmap v1449: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:15 smithi070 ceph-mon[29548]: pgmap v1450: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:15 smithi070 ceph-mon[39923]: pgmap v1450: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:15 smithi155 ceph-mon[32688]: pgmap v1450: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:17 smithi070 ceph-mon[29548]: pgmap v1451: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:17 smithi070 ceph-mon[39923]: pgmap v1451: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:17 smithi155 ceph-mon[32688]: pgmap v1451: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:19 smithi070 ceph-mon[29548]: pgmap v1452: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:19 smithi070 ceph-mon[39923]: pgmap v1452: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:19 smithi155 ceph-mon[32688]: pgmap v1452: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:20.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:04:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:04:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:21 smithi070 ceph-mon[29548]: pgmap v1453: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:21 smithi070 ceph-mon[39923]: pgmap v1453: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:21 smithi155 ceph-mon[32688]: pgmap v1453: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:04:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:04:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:04:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:23 smithi070 ceph-mon[29548]: pgmap v1454: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:23 smithi070 ceph-mon[39923]: pgmap v1454: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:23 smithi155 ceph-mon[32688]: pgmap v1454: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:25 smithi070 ceph-mon[29548]: pgmap v1455: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:25 smithi070 ceph-mon[39923]: pgmap v1455: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:25 smithi155 ceph-mon[32688]: pgmap v1455: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:27 smithi070 ceph-mon[29548]: pgmap v1456: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:27 smithi070 ceph-mon[39923]: pgmap v1456: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:27 smithi155 ceph-mon[32688]: pgmap v1456: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:29.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:28 smithi070 ceph-mon[29548]: pgmap v1457: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:29.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:28 smithi070 ceph-mon[39923]: pgmap v1457: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:28 smithi155 ceph-mon[32688]: pgmap v1457: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:30.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:04:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:04:31.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:31.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:31 smithi070 ceph-mon[39923]: pgmap v1458: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:31.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:31 smithi070 ceph-mon[29548]: pgmap v1458: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:31.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:31 smithi155 ceph-mon[32688]: pgmap v1458: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:04:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:04:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:04:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:33 smithi070 ceph-mon[29548]: pgmap v1459: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:33 smithi070 ceph-mon[39923]: pgmap v1459: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:33 smithi155 ceph-mon[32688]: pgmap v1459: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:35 smithi070 ceph-mon[29548]: pgmap v1460: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:35 smithi070 ceph-mon[39923]: pgmap v1460: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:35 smithi155 ceph-mon[32688]: pgmap v1460: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.544 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.545 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.546 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.547 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.548 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.549 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.550 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.551 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.552 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:04:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:37 smithi070 ceph-mon[29548]: pgmap v1461: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:37 smithi070 ceph-mon[39923]: pgmap v1461: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:37 smithi155 ceph-mon[32688]: pgmap v1461: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:39 smithi070 ceph-mon[29548]: pgmap v1462: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:39 smithi070 ceph-mon[39923]: pgmap v1462: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:39 smithi155 ceph-mon[32688]: pgmap v1462: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:04:40.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:04:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:04:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:41 smithi070 ceph-mon[29548]: pgmap v1463: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:41 smithi070 ceph-mon[39923]: pgmap v1463: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:41 smithi155 ceph-mon[32688]: pgmap v1463: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:04:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:04:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:04:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:43 smithi070 ceph-mon[39923]: pgmap v1464: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:43 smithi070 ceph-mon[29548]: pgmap v1464: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:43 smithi155 ceph-mon[32688]: pgmap v1464: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:45 smithi070 ceph-mon[29548]: pgmap v1465: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:45 smithi070 ceph-mon[39923]: pgmap v1465: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:45 smithi155 ceph-mon[32688]: pgmap v1465: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[39923]: pgmap v1466: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:04:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[29548]: pgmap v1466: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:04:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:47 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:47 smithi155 ceph-mon[32688]: pgmap v1466: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:47 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:04:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:47 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:47 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:48 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:04:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:48 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:04:48.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:48 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:48 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:04:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:48 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:04:48.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:48 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:48 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:04:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:48 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:04:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:48 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:04:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:49 smithi070 ceph-mon[29548]: pgmap v1467: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:49 smithi070 ceph-mon[39923]: pgmap v1467: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:49 smithi155 ceph-mon[32688]: pgmap v1467: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:50.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:04:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:04:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:51 smithi070 ceph-mon[29548]: pgmap v1468: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:51 smithi070 ceph-mon[39923]: pgmap v1468: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:04:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:51 smithi155 ceph-mon[32688]: pgmap v1468: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:04:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:04:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:04:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:53 smithi070 ceph-mon[29548]: pgmap v1469: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:53 smithi070 ceph-mon[39923]: pgmap v1469: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:53 smithi155 ceph-mon[32688]: pgmap v1469: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:55 smithi070 ceph-mon[29548]: pgmap v1470: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:55 smithi070 ceph-mon[39923]: pgmap v1470: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:55 smithi155 ceph-mon[32688]: pgmap v1470: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:04:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:04:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:57 smithi070 ceph-mon[29548]: pgmap v1471: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:57 smithi070 ceph-mon[39923]: pgmap v1471: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:57 smithi155 ceph-mon[32688]: pgmap v1471: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:04:59 smithi070 ceph-mon[29548]: pgmap v1472: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:04:59.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:04:59 smithi070 ceph-mon[39923]: pgmap v1472: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:04:59 smithi155 ceph-mon[32688]: pgmap v1472: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:00.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:05:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:05:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:01 smithi070 ceph-mon[29548]: pgmap v1473: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:01 smithi070 ceph-mon[39923]: pgmap v1473: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:01 smithi155 ceph-mon[32688]: pgmap v1473: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:05:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:05:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:05:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:03 smithi070 ceph-mon[29548]: pgmap v1474: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:03.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:03 smithi070 ceph-mon[39923]: pgmap v1474: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:03 smithi155 ceph-mon[32688]: pgmap v1474: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:05.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:05 smithi070 ceph-mon[39923]: pgmap v1475: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:05.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:05 smithi070 ceph-mon[29548]: pgmap v1475: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:05 smithi155 ceph-mon[32688]: pgmap v1475: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:07 smithi070 ceph-mon[29548]: pgmap v1476: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:07 smithi070 ceph-mon[39923]: pgmap v1476: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:07 smithi155 ceph-mon[32688]: pgmap v1476: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:09 smithi070 ceph-mon[29548]: pgmap v1477: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:09 smithi070 ceph-mon[39923]: pgmap v1477: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:09 smithi155 ceph-mon[32688]: pgmap v1477: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:10.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:05:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:05:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:11 smithi070 ceph-mon[29548]: pgmap v1478: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:11 smithi070 ceph-mon[39923]: pgmap v1478: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:11 smithi155 ceph-mon[32688]: pgmap v1478: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:05:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:05:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:05:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:13 smithi070 ceph-mon[29548]: pgmap v1479: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:13 smithi070 ceph-mon[39923]: pgmap v1479: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:13 smithi155 ceph-mon[32688]: pgmap v1479: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:15 smithi070 ceph-mon[29548]: pgmap v1480: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:15 smithi070 ceph-mon[39923]: pgmap v1480: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:15 smithi155 ceph-mon[32688]: pgmap v1480: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:17 smithi070 ceph-mon[29548]: pgmap v1481: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:17.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:17 smithi070 ceph-mon[39923]: pgmap v1481: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:17 smithi155 ceph-mon[32688]: pgmap v1481: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:19 smithi070 ceph-mon[29548]: pgmap v1482: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:05:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:19 smithi070 ceph-mon[39923]: pgmap v1482: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:05:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:19 smithi155 ceph-mon[32688]: pgmap v1482: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:05:20.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:05:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:05:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:21 smithi070 ceph-mon[29548]: pgmap v1483: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:21 smithi070 ceph-mon[39923]: pgmap v1483: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:21 smithi155 ceph-mon[32688]: pgmap v1483: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:05:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:05:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:05:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:23 smithi070 ceph-mon[29548]: pgmap v1484: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:23 smithi070 ceph-mon[39923]: pgmap v1484: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:23 smithi155 ceph-mon[32688]: pgmap v1484: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:25 smithi070 ceph-mon[29548]: pgmap v1485: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:25 smithi070 ceph-mon[39923]: pgmap v1485: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:25 smithi155 ceph-mon[32688]: pgmap v1485: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:27 smithi070 ceph-mon[29548]: pgmap v1486: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:27 smithi070 ceph-mon[39923]: pgmap v1486: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:27 smithi155 ceph-mon[32688]: pgmap v1486: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:29 smithi070 ceph-mon[29548]: pgmap v1487: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:05:29.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:29 smithi070 ceph-mon[39923]: pgmap v1487: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:05:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:29 smithi155 ceph-mon[32688]: pgmap v1487: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:05:30.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:05:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:05:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:31 smithi070 ceph-mon[29548]: pgmap v1488: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:31 smithi070 ceph-mon[39923]: pgmap v1488: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:31 smithi155 ceph-mon[32688]: pgmap v1488: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:05:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:05:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:05:33.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:33 smithi070 ceph-mon[39923]: pgmap v1489: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:33.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:33 smithi070 ceph-mon[29548]: pgmap v1489: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:33 smithi155 ceph-mon[32688]: pgmap v1489: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:35 smithi070 ceph-mon[29548]: pgmap v1490: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:35 smithi070 ceph-mon[39923]: pgmap v1490: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:35 smithi155 ceph-mon[32688]: pgmap v1490: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:37 smithi070 ceph-mon[29548]: pgmap v1491: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:37 smithi070 ceph-mon[39923]: pgmap v1491: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:37 smithi155 ceph-mon[32688]: pgmap v1491: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:39 smithi070 ceph-mon[29548]: pgmap v1492: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:39 smithi070 ceph-mon[39923]: pgmap v1492: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:39 smithi155 ceph-mon[32688]: pgmap v1492: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:40.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:05:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:05:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:41 smithi070 ceph-mon[29548]: pgmap v1493: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:41 smithi070 ceph-mon[39923]: pgmap v1493: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:41.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:41 smithi155 ceph-mon[32688]: pgmap v1493: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:05:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:05:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:05:43.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:43 smithi070 ceph-mon[39923]: pgmap v1494: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:43 smithi070 ceph-mon[29548]: pgmap v1494: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:43 smithi155 ceph-mon[32688]: pgmap v1494: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:45 smithi155 ceph-mon[32688]: pgmap v1495: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:46.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:45 smithi070 ceph-mon[29548]: pgmap v1495: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:46.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:45 smithi070 ceph-mon[39923]: pgmap v1495: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:47 smithi155 ceph-mon[32688]: pgmap v1496: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:47 smithi070 ceph-mon[29548]: pgmap v1496: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:47 smithi070 ceph-mon[39923]: pgmap v1496: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:49.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:48 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:05:49.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:48 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:05:49.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:48 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:05:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:49 smithi155 ceph-mon[32688]: pgmap v1497: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:49 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:05:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:49 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:05:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:49 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:05:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[29548]: pgmap v1497: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:05:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:05:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:05:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[39923]: pgmap v1497: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:05:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:05:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:49 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:05:50.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:05:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:05:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:51.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:50 smithi070 ceph-mon[29548]: pgmap v1498: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:50 smithi070 ceph-mon[39923]: pgmap v1498: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:05:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:50 smithi155 ceph-mon[32688]: pgmap v1498: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:05:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:05:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:05:53.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:53 smithi070 ceph-mon[39923]: pgmap v1499: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:53 smithi070 ceph-mon[29548]: pgmap v1499: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:53 smithi155 ceph-mon[32688]: pgmap v1499: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:55.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:55 smithi070 ceph-mon[39923]: pgmap v1500: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:55.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:55 smithi070 ceph-mon[29548]: pgmap v1500: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:55 smithi155 ceph-mon[32688]: pgmap v1500: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:05:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:57.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:05:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:57 smithi070 ceph-mon[29548]: pgmap v1501: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:57.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:57 smithi070 ceph-mon[39923]: pgmap v1501: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:57 smithi155 ceph-mon[32688]: pgmap v1501: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:05:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:05:59 smithi070 ceph-mon[29548]: pgmap v1502: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:05:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:05:59 smithi070 ceph-mon[39923]: pgmap v1502: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:06:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:05:59 smithi155 ceph-mon[32688]: pgmap v1502: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:06:00.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:06:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:06:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:01 smithi070 ceph-mon[29548]: pgmap v1503: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:01 smithi070 ceph-mon[39923]: pgmap v1503: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:01 smithi155 ceph-mon[32688]: pgmap v1503: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:06:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:06:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:06:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:03 smithi070 ceph-mon[29548]: pgmap v1504: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:03 smithi070 ceph-mon[39923]: pgmap v1504: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:03 smithi155 ceph-mon[32688]: pgmap v1504: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:05 smithi070 ceph-mon[29548]: pgmap v1505: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:05 smithi070 ceph-mon[39923]: pgmap v1505: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:06.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:05 smithi155 ceph-mon[32688]: pgmap v1505: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:07 smithi070 ceph-mon[29548]: pgmap v1506: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:07 smithi070 ceph-mon[39923]: pgmap v1506: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:07 smithi155 ceph-mon[32688]: pgmap v1506: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:09 smithi070 ceph-mon[29548]: pgmap v1507: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:09 smithi070 ceph-mon[39923]: pgmap v1507: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:09 smithi155 ceph-mon[32688]: pgmap v1507: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:10.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:06:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:06:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:11 smithi070 ceph-mon[29548]: pgmap v1508: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:11 smithi070 ceph-mon[39923]: pgmap v1508: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:11 smithi155 ceph-mon[32688]: pgmap v1508: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:12.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:06:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:06:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:06:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:13 smithi070 ceph-mon[29548]: pgmap v1509: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:13 smithi070 ceph-mon[39923]: pgmap v1509: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:13 smithi155 ceph-mon[32688]: pgmap v1509: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:15 smithi070 ceph-mon[29548]: pgmap v1510: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:15 smithi070 ceph-mon[39923]: pgmap v1510: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:15 smithi155 ceph-mon[32688]: pgmap v1510: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:17.555 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.556 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.557 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.558 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.559 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.560 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.561 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.562 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.563 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.563 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.563 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.563 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.563 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.563 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.563 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:06:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:17 smithi070 ceph-mon[29548]: pgmap v1511: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:17 smithi070 ceph-mon[39923]: pgmap v1511: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:17 smithi155 ceph-mon[32688]: pgmap v1511: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:19.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:19 smithi070 ceph-mon[39923]: pgmap v1512: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:19 smithi070 ceph-mon[29548]: pgmap v1512: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:19 smithi155 ceph-mon[32688]: pgmap v1512: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:20.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:06:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:06:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:21.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:21 smithi070 ceph-mon[39923]: pgmap v1513: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:21 smithi070 ceph-mon[29548]: pgmap v1513: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:21 smithi155 ceph-mon[32688]: pgmap v1513: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:06:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:06:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:06:23.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:23 smithi070 ceph-mon[39923]: pgmap v1514: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:23 smithi070 ceph-mon[29548]: pgmap v1514: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:23 smithi155 ceph-mon[32688]: pgmap v1514: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:25 smithi070 ceph-mon[29548]: pgmap v1515: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:25 smithi070 ceph-mon[39923]: pgmap v1515: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:25 smithi155 ceph-mon[32688]: pgmap v1515: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:27.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:27 smithi070 ceph-mon[29548]: pgmap v1516: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:27 smithi070 ceph-mon[39923]: pgmap v1516: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:27 smithi155 ceph-mon[32688]: pgmap v1516: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:29 smithi070 ceph-mon[29548]: pgmap v1517: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:29 smithi070 ceph-mon[39923]: pgmap v1517: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:29 smithi155 ceph-mon[32688]: pgmap v1517: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:30.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:06:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:06:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:31 smithi070 ceph-mon[29548]: pgmap v1518: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:31 smithi070 ceph-mon[39923]: pgmap v1518: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:31 smithi155 ceph-mon[32688]: pgmap v1518: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:06:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:06:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:06:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:33 smithi070 ceph-mon[29548]: pgmap v1519: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:33.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:33 smithi070 ceph-mon[39923]: pgmap v1519: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:34.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:33 smithi155 ceph-mon[32688]: pgmap v1519: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:35 smithi070 ceph-mon[29548]: pgmap v1520: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:35 smithi070 ceph-mon[39923]: pgmap v1520: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:35 smithi155 ceph-mon[32688]: pgmap v1520: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:37 smithi070 ceph-mon[29548]: pgmap v1521: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:37 smithi070 ceph-mon[39923]: pgmap v1521: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:37 smithi155 ceph-mon[32688]: pgmap v1521: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:39.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:39 smithi070 ceph-mon[29548]: pgmap v1522: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:06:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:39 smithi070 ceph-mon[39923]: pgmap v1522: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:06:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:39 smithi155 ceph-mon[32688]: pgmap v1522: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:06:40.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:06:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:06:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:41 smithi070 ceph-mon[29548]: pgmap v1523: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:41 smithi070 ceph-mon[39923]: pgmap v1523: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:41 smithi155 ceph-mon[32688]: pgmap v1523: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:06:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:06:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:06:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:43 smithi070 ceph-mon[29548]: pgmap v1524: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:43 smithi070 ceph-mon[39923]: pgmap v1524: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:43 smithi155 ceph-mon[32688]: pgmap v1524: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:45.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:45 smithi070 ceph-mon[29548]: pgmap v1525: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:45 smithi070 ceph-mon[39923]: pgmap v1525: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:45 smithi155 ceph-mon[32688]: pgmap v1525: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:47 smithi070 ceph-mon[29548]: pgmap v1526: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:47 smithi070 ceph-mon[39923]: pgmap v1526: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:47 smithi155 ceph-mon[32688]: pgmap v1526: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:49 smithi155 ceph-mon[32688]: pgmap v1527: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:50.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:49 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:06:50.163 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:49 smithi070 ceph-mon[29548]: pgmap v1527: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:50.163 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:49 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:06:50.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:49 smithi070 ceph-mon[39923]: pgmap v1527: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:50.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:49 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:06:50.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:06:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:06:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:52.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:51 smithi155 ceph-mon[32688]: pgmap v1528: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:52.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:52.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:51 smithi070 ceph-mon[29548]: pgmap v1528: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:52.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:06:52.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:51 smithi070 ceph-mon[39923]: pgmap v1528: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:52.325 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:06:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:06:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:06:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[29548]: pgmap v1529: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[39923]: pgmap v1529: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:06:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:06:53.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:53 smithi155 ceph-mon[32688]: pgmap v1529: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:06:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:06:54.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:06:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:55 smithi155 ceph-mon[32688]: pgmap v1530: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:55 smithi070 ceph-mon[29548]: pgmap v1530: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:55 smithi070 ceph-mon[39923]: pgmap v1530: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:06:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:06:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:57 smithi070 ceph-mon[29548]: pgmap v1531: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:57 smithi070 ceph-mon[39923]: pgmap v1531: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:06:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:57 smithi155 ceph-mon[32688]: pgmap v1531: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:06:59 smithi070 ceph-mon[29548]: pgmap v1532: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:07:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:06:59 smithi070 ceph-mon[39923]: pgmap v1532: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:07:00.253 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:06:59 smithi155 ceph-mon[32688]: pgmap v1532: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:07:00.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:07:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:07:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:00 smithi070 ceph-mon[29548]: pgmap v1533: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:00 smithi070 ceph-mon[39923]: pgmap v1533: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:00 smithi155 ceph-mon[32688]: pgmap v1533: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:07:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:07:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:07:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:03 smithi070 ceph-mon[29548]: pgmap v1534: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:03 smithi070 ceph-mon[39923]: pgmap v1534: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:03 smithi155 ceph-mon[32688]: pgmap v1534: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:05 smithi070 ceph-mon[29548]: pgmap v1535: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:05 smithi070 ceph-mon[39923]: pgmap v1535: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:05 smithi155 ceph-mon[32688]: pgmap v1535: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:07 smithi070 ceph-mon[29548]: pgmap v1536: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:07 smithi070 ceph-mon[39923]: pgmap v1536: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:07 smithi155 ceph-mon[32688]: pgmap v1536: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:09 smithi070 ceph-mon[29548]: pgmap v1537: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:09 smithi070 ceph-mon[39923]: pgmap v1537: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:09 smithi155 ceph-mon[32688]: pgmap v1537: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:10.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:07:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:07:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:11 smithi070 ceph-mon[29548]: pgmap v1538: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:11 smithi070 ceph-mon[39923]: pgmap v1538: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:11 smithi155 ceph-mon[32688]: pgmap v1538: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:07:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:07:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:07:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:13 smithi070 ceph-mon[29548]: pgmap v1539: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:13.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:13 smithi070 ceph-mon[39923]: pgmap v1539: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:13 smithi155 ceph-mon[32688]: pgmap v1539: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:15 smithi070 ceph-mon[29548]: pgmap v1540: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:15 smithi070 ceph-mon[39923]: pgmap v1540: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:15 smithi155 ceph-mon[32688]: pgmap v1540: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:17 smithi070 ceph-mon[29548]: pgmap v1541: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:17 smithi070 ceph-mon[39923]: pgmap v1541: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:17 smithi155 ceph-mon[32688]: pgmap v1541: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:19 smithi070 ceph-mon[29548]: pgmap v1542: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:19 smithi070 ceph-mon[39923]: pgmap v1542: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:19 smithi155 ceph-mon[32688]: pgmap v1542: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:20.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:07:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:07:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:21 smithi070 ceph-mon[29548]: pgmap v1543: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:21 smithi070 ceph-mon[39923]: pgmap v1543: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:21 smithi155 ceph-mon[32688]: pgmap v1543: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:07:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:07:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:07:23.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:23 smithi070 ceph-mon[29548]: pgmap v1544: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:23 smithi070 ceph-mon[39923]: pgmap v1544: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:23 smithi155 ceph-mon[32688]: pgmap v1544: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:25 smithi070 ceph-mon[29548]: pgmap v1545: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:25 smithi070 ceph-mon[39923]: pgmap v1545: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:25 smithi155 ceph-mon[32688]: pgmap v1545: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:26.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:26.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:27 smithi070 ceph-mon[29548]: pgmap v1546: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:27 smithi070 ceph-mon[39923]: pgmap v1546: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:27 smithi155 ceph-mon[32688]: pgmap v1546: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:29 smithi070 ceph-mon[29548]: pgmap v1547: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:29 smithi070 ceph-mon[39923]: pgmap v1547: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:29 smithi155 ceph-mon[32688]: pgmap v1547: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:30.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:07:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:07:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:31 smithi070 ceph-mon[29548]: pgmap v1548: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:31 smithi070 ceph-mon[39923]: pgmap v1548: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:31 smithi155 ceph-mon[32688]: pgmap v1548: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:07:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:07:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:07:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:33 smithi070 ceph-mon[29548]: pgmap v1549: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:33 smithi070 ceph-mon[39923]: pgmap v1549: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:33 smithi155 ceph-mon[32688]: pgmap v1549: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:35 smithi070 ceph-mon[29548]: pgmap v1550: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:35 smithi070 ceph-mon[39923]: pgmap v1550: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:35 smithi155 ceph-mon[32688]: pgmap v1550: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:37 smithi070 ceph-mon[29548]: pgmap v1551: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:37 smithi070 ceph-mon[39923]: pgmap v1551: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:37 smithi155 ceph-mon[32688]: pgmap v1551: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:39 smithi070 ceph-mon[29548]: pgmap v1552: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:39 smithi070 ceph-mon[39923]: pgmap v1552: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:39 smithi155 ceph-mon[32688]: pgmap v1552: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:40.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:07:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:07:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:41 smithi070 ceph-mon[29548]: pgmap v1553: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:41 smithi070 ceph-mon[39923]: pgmap v1553: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:41 smithi155 ceph-mon[32688]: pgmap v1553: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:07:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:07:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:07:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:43 smithi070 ceph-mon[29548]: pgmap v1554: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:43 smithi070 ceph-mon[39923]: pgmap v1554: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:43 smithi155 ceph-mon[32688]: pgmap v1554: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:45 smithi070 ceph-mon[29548]: pgmap v1555: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:45 smithi070 ceph-mon[39923]: pgmap v1555: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:45 smithi155 ceph-mon[32688]: pgmap v1555: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:47 smithi070 ceph-mon[29548]: pgmap v1556: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:47 smithi070 ceph-mon[39923]: pgmap v1556: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:47 smithi155 ceph-mon[32688]: pgmap v1556: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:49.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:49 smithi070 ceph-mon[29548]: pgmap v1557: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:49 smithi070 ceph-mon[39923]: pgmap v1557: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:49 smithi155 ceph-mon[32688]: pgmap v1557: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:50.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:07:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:07:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:51 smithi070 ceph-mon[29548]: pgmap v1558: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:51 smithi070 ceph-mon[39923]: pgmap v1558: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:07:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:51 smithi155 ceph-mon[32688]: pgmap v1558: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:07:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:07:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:07:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:53 smithi070 ceph-mon[29548]: pgmap v1559: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:53 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:07:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:53 smithi070 ceph-mon[39923]: pgmap v1559: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:53 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:07:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:53 smithi155 ceph-mon[32688]: pgmap v1559: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:53 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:07:55.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:55 smithi155 ceph-mon[32688]: pgmap v1560: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:55 smithi070 ceph-mon[29548]: pgmap v1560: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:56.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:55 smithi070 ceph-mon[39923]: pgmap v1560: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:07:56.567 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.567 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.567 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.568 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.569 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.570 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.571 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.572 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.573 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.574 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:07:56.955 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:07:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:57 smithi155 ceph-mon[32688]: pgmap v1561: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:07:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:07:58.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[29548]: pgmap v1561: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[39923]: pgmap v1561: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:07:58.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:07:58.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:07:58.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:08:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:07:59 smithi155 ceph-mon[32688]: pgmap v1562: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:07:59 smithi070 ceph-mon[29548]: pgmap v1562: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:07:59 smithi070 ceph-mon[39923]: pgmap v1562: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:00.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:08:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:08:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:02.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:01 smithi155 ceph-mon[32688]: pgmap v1563: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:02.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:02.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:01 smithi070 ceph-mon[29548]: pgmap v1563: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:02.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:02.061 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:01 smithi070 ceph-mon[39923]: pgmap v1563: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:08:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:08:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:08:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:03 smithi070 ceph-mon[29548]: pgmap v1564: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:03 smithi070 ceph-mon[39923]: pgmap v1564: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:03 smithi155 ceph-mon[32688]: pgmap v1564: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:05 smithi070 ceph-mon[29548]: pgmap v1565: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:05 smithi070 ceph-mon[39923]: pgmap v1565: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:05 smithi155 ceph-mon[32688]: pgmap v1565: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:07 smithi070 ceph-mon[29548]: pgmap v1566: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:07 smithi070 ceph-mon[39923]: pgmap v1566: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:07 smithi155 ceph-mon[32688]: pgmap v1566: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:09 smithi070 ceph-mon[29548]: pgmap v1567: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:09 smithi070 ceph-mon[39923]: pgmap v1567: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:09 smithi155 ceph-mon[32688]: pgmap v1567: 292 pgs: 292 active+clean; 582 KiB data, 790 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:10.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:08:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:08:12.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:12.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:11 smithi070 ceph-mon[29548]: pgmap v1568: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:12.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:11 smithi070 ceph-mon[39923]: pgmap v1568: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:12.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:11 smithi155 ceph-mon[32688]: pgmap v1568: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:12.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:08:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:08:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:08:14.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:13 smithi070 ceph-mon[29548]: pgmap v1569: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:14.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:13 smithi070 ceph-mon[39923]: pgmap v1569: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:13 smithi155 ceph-mon[32688]: pgmap v1569: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:15 smithi070 ceph-mon[29548]: pgmap v1570: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:15 smithi070 ceph-mon[39923]: pgmap v1570: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:15 smithi155 ceph-mon[32688]: pgmap v1570: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:17.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:16 smithi070 ceph-mon[29548]: pgmap v1571: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:17.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:16 smithi070 ceph-mon[39923]: pgmap v1571: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:16 smithi155 ceph-mon[32688]: pgmap v1571: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:19 smithi070 ceph-mon[29548]: pgmap v1572: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:19 smithi070 ceph-mon[39923]: pgmap v1572: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:19 smithi155 ceph-mon[32688]: pgmap v1572: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:20.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:08:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:08:21.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:21 smithi070 ceph-mon[29548]: pgmap v1573: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:21 smithi070 ceph-mon[39923]: pgmap v1573: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:22.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:22.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:21 smithi155 ceph-mon[32688]: pgmap v1573: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:08:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:08:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:08:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:08:23.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:23 smithi070 ceph-mon[29548]: pgmap v1574: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:23.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:23 smithi070 ceph-mon[39923]: pgmap v1574: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:23 smithi155 ceph-mon[32688]: pgmap v1574: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:25.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:25 smithi070 ceph-mon[29548]: pgmap v1575: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:25.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:25 smithi070 ceph-mon[39923]: pgmap v1575: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:25 smithi155 ceph-mon[32688]: pgmap v1575: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:26.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:26.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:27.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:27 smithi070 ceph-mon[29548]: pgmap v1576: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:27.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:27 smithi070 ceph-mon[39923]: pgmap v1576: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:28.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:27 smithi155 ceph-mon[32688]: pgmap v1576: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:29 smithi070 ceph-mon[29548]: pgmap v1577: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:29 smithi070 ceph-mon[39923]: pgmap v1577: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:29 smithi155 ceph-mon[32688]: pgmap v1577: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:30.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:08:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:08:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:31 smithi070 ceph-mon[29548]: pgmap v1578: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:31 smithi070 ceph-mon[39923]: pgmap v1578: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:31 smithi155 ceph-mon[32688]: pgmap v1578: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:08:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:08:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:08:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:33 smithi070 ceph-mon[29548]: pgmap v1579: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:33 smithi070 ceph-mon[39923]: pgmap v1579: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:33 smithi155 ceph-mon[32688]: pgmap v1579: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:35.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:35 smithi070 ceph-mon[29548]: pgmap v1580: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:35 smithi070 ceph-mon[39923]: pgmap v1580: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:35 smithi155 ceph-mon[32688]: pgmap v1580: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:37 smithi070 ceph-mon[29548]: pgmap v1581: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:37 smithi070 ceph-mon[39923]: pgmap v1581: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:37 smithi155 ceph-mon[32688]: pgmap v1581: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:39.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:39 smithi070 ceph-mon[29548]: pgmap v1582: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:08:39.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:39 smithi070 ceph-mon[39923]: pgmap v1582: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:08:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:39 smithi155 ceph-mon[32688]: pgmap v1582: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:08:40.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:08:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:08:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:41 smithi070 ceph-mon[29548]: pgmap v1583: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:41.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:41 smithi070 ceph-mon[39923]: pgmap v1583: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:41 smithi155 ceph-mon[32688]: pgmap v1583: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:42.492 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:08:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:08:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:08:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:43 smithi070 ceph-mon[29548]: pgmap v1584: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:43 smithi070 ceph-mon[39923]: pgmap v1584: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:43 smithi155 ceph-mon[32688]: pgmap v1584: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:45 smithi070 ceph-mon[29548]: pgmap v1585: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:45 smithi070 ceph-mon[39923]: pgmap v1585: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:45 smithi155 ceph-mon[32688]: pgmap v1585: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:47.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:47 smithi070 ceph-mon[29548]: pgmap v1586: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:47.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:47 smithi070 ceph-mon[39923]: pgmap v1586: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:48.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:47 smithi155 ceph-mon[32688]: pgmap v1586: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:49.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:49 smithi070 ceph-mon[39923]: pgmap v1587: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:49.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:49 smithi070 ceph-mon[29548]: pgmap v1587: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:49 smithi155 ceph-mon[32688]: pgmap v1587: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:08:50.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:08:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:08:51.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:51 smithi070 ceph-mon[39923]: pgmap v1588: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:51.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:51 smithi070 ceph-mon[29548]: pgmap v1588: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:08:52.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:51 smithi155 ceph-mon[32688]: pgmap v1588: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:08:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:08:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:08:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:53 smithi155 ceph-mon[32688]: pgmap v1589: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:54.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:53 smithi070 ceph-mon[39923]: pgmap v1589: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:54.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:53 smithi070 ceph-mon[29548]: pgmap v1589: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:55 smithi155 ceph-mon[32688]: pgmap v1590: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:55 smithi070 ceph-mon[29548]: pgmap v1590: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:55 smithi070 ceph-mon[39923]: pgmap v1590: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:08:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:57.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:08:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:57 smithi155 ceph-mon[32688]: pgmap v1591: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:08:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:57 smithi070 ceph-mon[29548]: pgmap v1591: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:08:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:57 smithi070 ceph-mon[39923]: pgmap v1591: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:08:59.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[29548]: pgmap v1592: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[39923]: pgmap v1592: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:08:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:08:59.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:08:59 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:09:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:59 smithi155 ceph-mon[32688]: pgmap v1592: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:59 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:09:00.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:59 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:09:00.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:59 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:09:00.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:59 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:09:00.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:08:59 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:09:00.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:09:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:09:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:01 smithi070 ceph-mon[39923]: pgmap v1593: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:01.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:01 smithi070 ceph-mon[29548]: pgmap v1593: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:01 smithi155 ceph-mon[32688]: pgmap v1593: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:02.492 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:09:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:09:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:09:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:03 smithi070 ceph-mon[29548]: pgmap v1594: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:03 smithi070 ceph-mon[39923]: pgmap v1594: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:03 smithi155 ceph-mon[32688]: pgmap v1594: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:05 smithi070 ceph-mon[29548]: pgmap v1595: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:05 smithi070 ceph-mon[39923]: pgmap v1595: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:05 smithi155 ceph-mon[32688]: pgmap v1595: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:07.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:07 smithi070 ceph-mon[39923]: pgmap v1596: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:07.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:07 smithi070 ceph-mon[29548]: pgmap v1596: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:07 smithi155 ceph-mon[32688]: pgmap v1596: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:09.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:09 smithi070 ceph-mon[39923]: pgmap v1597: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:09.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:09 smithi070 ceph-mon[29548]: pgmap v1597: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:09 smithi155 ceph-mon[32688]: pgmap v1597: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:10.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:09:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:09:12.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:12.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:11 smithi155 ceph-mon[32688]: pgmap v1598: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:12.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:12.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:12.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:11 smithi070 ceph-mon[29548]: pgmap v1598: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:12.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:12.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:12.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:11 smithi070 ceph-mon[39923]: pgmap v1598: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:12.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:09:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:09:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:09:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:13 smithi155 ceph-mon[32688]: pgmap v1599: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:14.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:13 smithi070 ceph-mon[29548]: pgmap v1599: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:14.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:13 smithi070 ceph-mon[39923]: pgmap v1599: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:15 smithi155 ceph-mon[32688]: pgmap v1600: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:15 smithi070 ceph-mon[29548]: pgmap v1600: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:15 smithi070 ceph-mon[39923]: pgmap v1600: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:17 smithi155 ceph-mon[32688]: pgmap v1601: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:18.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:17 smithi070 ceph-mon[29548]: pgmap v1601: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:18.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:17 smithi070 ceph-mon[39923]: pgmap v1601: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:20.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:19 smithi070 ceph-mon[39923]: pgmap v1602: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:20.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:19 smithi070 ceph-mon[29548]: pgmap v1602: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:19 smithi155 ceph-mon[32688]: pgmap v1602: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:20.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:09:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:09:22.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:22.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:21 smithi070 ceph-mon[29548]: pgmap v1603: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:22.044 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:22.044 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:21 smithi070 ceph-mon[39923]: pgmap v1603: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:21 smithi155 ceph-mon[32688]: pgmap v1603: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:09:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:09:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:09:24.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:23 smithi070 ceph-mon[29548]: pgmap v1604: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:24.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:23 smithi070 ceph-mon[39923]: pgmap v1604: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:24.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:23 smithi155 ceph-mon[32688]: pgmap v1604: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:26.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:25 smithi070 ceph-mon[29548]: pgmap v1605: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:25 smithi070 ceph-mon[39923]: pgmap v1605: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:25 smithi155 ceph-mon[32688]: pgmap v1605: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:28.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:27 smithi070 ceph-mon[39923]: pgmap v1606: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:28.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:27 smithi070 ceph-mon[29548]: pgmap v1606: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:27 smithi155 ceph-mon[32688]: pgmap v1606: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:29 smithi070 ceph-mon[29548]: pgmap v1607: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:30.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:29 smithi070 ceph-mon[39923]: pgmap v1607: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:29 smithi155 ceph-mon[32688]: pgmap v1607: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:30.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:09:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:09:32.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:32.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:31 smithi070 ceph-mon[29548]: pgmap v1608: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:31 smithi070 ceph-mon[39923]: pgmap v1608: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:31 smithi155 ceph-mon[32688]: pgmap v1608: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:09:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:09:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:09:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:33 smithi070 ceph-mon[29548]: pgmap v1609: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:33 smithi070 ceph-mon[39923]: pgmap v1609: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:33 smithi155 ceph-mon[32688]: pgmap v1609: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:36.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:35 smithi070 ceph-mon[39923]: pgmap v1610: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:36.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:35 smithi070 ceph-mon[29548]: pgmap v1610: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:36.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:35 smithi155 ceph-mon[32688]: pgmap v1610: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.579 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.580 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.581 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.582 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.583 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.584 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.585 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:36.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:09:38.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:37 smithi155 ceph-mon[32688]: pgmap v1611: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:38.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:37 smithi070 ceph-mon[39923]: pgmap v1611: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:37 smithi070 ceph-mon[29548]: pgmap v1611: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:39.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:38 smithi070 ceph-mon[29548]: pgmap v1612: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:39.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:38 smithi070 ceph-mon[39923]: pgmap v1612: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:39.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:38 smithi155 ceph-mon[32688]: pgmap v1612: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:40.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:09:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:09:41.505 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:09:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a[63394]: logger=cleanup t=2024-02-10T08:09:41.020420918Z level=info msg="Completed cleanup jobs" duration=7.292383ms 2024-02-10T08:09:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:41.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:41 smithi070 ceph-mon[29548]: pgmap v1613: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:41.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:41 smithi070 ceph-mon[39923]: pgmap v1613: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:42.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:42.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:41 smithi155 ceph-mon[32688]: pgmap v1613: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:09:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:09:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:09:42.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:42.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:43.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:43.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:43 smithi070 ceph-mon[29548]: pgmap v1614: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:43 smithi070 ceph-mon[39923]: pgmap v1614: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:43 smithi155 ceph-mon[32688]: pgmap v1614: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:45.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:45 smithi070 ceph-mon[29548]: pgmap v1615: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:45.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:45 smithi070 ceph-mon[39923]: pgmap v1615: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:46.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:45 smithi155 ceph-mon[32688]: pgmap v1615: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:47.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:47 smithi070 ceph-mon[39923]: pgmap v1616: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:47.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:47 smithi070 ceph-mon[29548]: pgmap v1616: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:48.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:47 smithi155 ceph-mon[32688]: pgmap v1616: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:49.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:49 smithi070 ceph-mon[29548]: pgmap v1617: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:49.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:49 smithi070 ceph-mon[39923]: pgmap v1617: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:50.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:49 smithi155 ceph-mon[32688]: pgmap v1617: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:50.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:09:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:09:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:51.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:51 smithi070 ceph-mon[29548]: pgmap v1618: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:51.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:51 smithi070 ceph-mon[39923]: pgmap v1618: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:09:52.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:51 smithi155 ceph-mon[32688]: pgmap v1618: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:09:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:09:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:09:53.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:53 smithi070 ceph-mon[29548]: pgmap v1619: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:53.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:53 smithi070 ceph-mon[39923]: pgmap v1619: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:54.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:53 smithi155 ceph-mon[32688]: pgmap v1619: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:55.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:55 smithi070 ceph-mon[29548]: pgmap v1620: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:55.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:55 smithi070 ceph-mon[39923]: pgmap v1620: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:56.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:55 smithi155 ceph-mon[32688]: pgmap v1620: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:09:56.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:56.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:57.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:09:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:57 smithi070 ceph-mon[29548]: pgmap v1621: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:57 smithi070 ceph-mon[39923]: pgmap v1621: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:57 smithi155 ceph-mon[32688]: pgmap v1621: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:09:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:59 smithi070 ceph-mon[39923]: pgmap v1622: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:09:59.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:09:59 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:09:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:59 smithi070 ceph-mon[29548]: pgmap v1622: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:09:59.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:09:59 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:10:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:59 smithi155 ceph-mon[32688]: pgmap v1622: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:10:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:09:59 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:10:00.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[29548]: pgmap v1623: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:01.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:10:01.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:10:01.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T08:10:01.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:01.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:00 smithi070 ceph-mon[39923]: pgmap v1623: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:10:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:10:01.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:10:01.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-10T08:10:01.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:01.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:00 smithi155 ceph-mon[32688]: pgmap v1623: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:10:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:10:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:10:03.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:03 smithi070 ceph-mon[29548]: pgmap v1624: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:03.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:03 smithi070 ceph-mon[39923]: pgmap v1624: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:03 smithi155 ceph-mon[32688]: pgmap v1624: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:05.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:05 smithi070 ceph-mon[29548]: pgmap v1625: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:05.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:05 smithi070 ceph-mon[39923]: pgmap v1625: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:05 smithi155 ceph-mon[32688]: pgmap v1625: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:07.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:07 smithi070 ceph-mon[29548]: pgmap v1626: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:07.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:07 smithi070 ceph-mon[39923]: pgmap v1626: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:07 smithi155 ceph-mon[32688]: pgmap v1626: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:09.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:09 smithi070 ceph-mon[29548]: pgmap v1627: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:10:09.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:09 smithi070 ceph-mon[39923]: pgmap v1627: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:10:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:09 smithi155 ceph-mon[32688]: pgmap v1627: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:10:10.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:10:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:10:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:11.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:11 smithi070 ceph-mon[29548]: pgmap v1628: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:11.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:11 smithi070 ceph-mon[39923]: pgmap v1628: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:11.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:12.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:12.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:11 smithi155 ceph-mon[32688]: pgmap v1628: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:12.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:12.492 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:10:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:10:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:10:13.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:13 smithi070 ceph-mon[29548]: pgmap v1629: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:13.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:13 smithi070 ceph-mon[39923]: pgmap v1629: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:14.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:13 smithi155 ceph-mon[32688]: pgmap v1629: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:15.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:15 smithi070 ceph-mon[29548]: pgmap v1630: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:15.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:15 smithi070 ceph-mon[39923]: pgmap v1630: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:16.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:15 smithi155 ceph-mon[32688]: pgmap v1630: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:17.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:17 smithi070 ceph-mon[29548]: pgmap v1631: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:17.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:17 smithi070 ceph-mon[39923]: pgmap v1631: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:18.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:17 smithi155 ceph-mon[32688]: pgmap v1631: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:19.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:19 smithi070 ceph-mon[29548]: pgmap v1632: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:19.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:19 smithi070 ceph-mon[39923]: pgmap v1632: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:20.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:19 smithi155 ceph-mon[32688]: pgmap v1632: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:20.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:10:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:10:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:21.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:21 smithi070 ceph-mon[39923]: pgmap v1633: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:21.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:21 smithi070 ceph-mon[29548]: pgmap v1633: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:22.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:22.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:21 smithi155 ceph-mon[32688]: pgmap v1633: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:10:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:10:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:10:23.993 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:23 smithi070 ceph-mon[29548]: pgmap v1634: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:23.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:23 smithi070 ceph-mon[39923]: pgmap v1634: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:24.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:23 smithi155 ceph-mon[32688]: pgmap v1634: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:26.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:25 smithi155 ceph-mon[32688]: pgmap v1635: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:26.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:25 smithi070 ceph-mon[29548]: pgmap v1635: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:25 smithi070 ceph-mon[39923]: pgmap v1635: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:27.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:27.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:28.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:27 smithi070 ceph-mon[39923]: pgmap v1636: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:28.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:27 smithi070 ceph-mon[39923]: osdmap e753: 8 total, 8 up, 8 in 2024-02-10T08:10:28.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:27 smithi070 ceph-mon[29548]: pgmap v1636: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:28.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:27 smithi070 ceph-mon[29548]: osdmap e753: 8 total, 8 up, 8 in 2024-02-10T08:10:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:27 smithi155 ceph-mon[32688]: pgmap v1636: 292 pgs: 292 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:10:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:27 smithi155 ceph-mon[32688]: osdmap e753: 8 total, 8 up, 8 in 2024-02-10T08:10:29.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:28 smithi070 ceph-mon[39923]: osdmap e754: 8 total, 8 up, 8 in 2024-02-10T08:10:29.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:28 smithi070 ceph-mon[39923]: osdmap e755: 8 total, 8 up, 8 in 2024-02-10T08:10:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:28 smithi070 ceph-mon[29548]: osdmap e754: 8 total, 8 up, 8 in 2024-02-10T08:10:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:28 smithi070 ceph-mon[29548]: osdmap e755: 8 total, 8 up, 8 in 2024-02-10T08:10:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:28 smithi155 ceph-mon[32688]: osdmap e754: 8 total, 8 up, 8 in 2024-02-10T08:10:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:28 smithi155 ceph-mon[32688]: osdmap e755: 8 total, 8 up, 8 in 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.586 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.587 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.588 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 1 sec 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: started 512 aios 2024-02-10T08:10:29.589 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b25ea5720 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2627ad70 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2627cd90 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2627bf20 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262808a0 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2627bd80 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26280d70 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281270 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281790 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281930 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281af0 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281c20 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281ee0 2024-02-10T08:10:29.590 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262820c0 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26282670 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262821d0 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262822e0 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26282430 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262827a0 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26282c80 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26282de0 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262833b0 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262828d0 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26282a00 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26282f10 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283040 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26282b70 2024-02-10T08:10:29.591 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283190 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283960 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262834c0 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283610 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283720 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283830 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283f90 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283af0 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283c40 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26283d70 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262840c0 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284530 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284660 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284c50 2024-02-10T08:10:29.592 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262841d0 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284320 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262847b0 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262848c0 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262849f0 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284b40 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285220 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284d80 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284e90 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26284fc0 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262850d0 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262857d0 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285330 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285440 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285590 2024-02-10T08:10:29.593 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285900 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285d90 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285f10 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262856c0 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286040 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281410 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281520 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26281630 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286610 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286170 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262862a0 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262863f0 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286720 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286920 2024-02-10T08:10:29.594 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286ef0 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286a50 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286b60 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286c90 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26286dc0 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262874e0 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287040 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287170 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262872a0 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262873d0 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287ad0 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287630 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287740 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287850 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287960 2024-02-10T08:10:29.595 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262880c0 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287c20 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287d70 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26287ea0 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262881d0 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288680 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262887d0 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288da0 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288320 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288450 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288900 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288a30 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288b60 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288c90 2024-02-10T08:10:29.596 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289370 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288ed0 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289000 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289130 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289260 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26288560 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262894a0 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262895f0 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289be0 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289740 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289870 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262899a0 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289ad0 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a1b0 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289d10 2024-02-10T08:10:29.597 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289e60 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26289f90 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a300 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a790 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a8e0 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628aed0 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a430 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a560 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628aa30 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ab40 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ac70 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ada0 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b480 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628afe0 2024-02-10T08:10:29.598 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b110 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a670 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b240 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b370 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b5d0 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628baa0 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628bbf0 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628c1c0 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b720 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b830 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628b960 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628c2f0 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628bd20 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628be70 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628bf80 2024-02-10T08:10:29.599 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628c0b0 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628a0a0 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628c440 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628c8d0 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ca20 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285a30 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26285b80 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d010 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628cb70 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628cc80 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628cdb0 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628cec0 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d600 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d160 2024-02-10T08:10:29.600 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d2b0 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d3e0 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d750 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628dbe0 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628dd10 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628c550 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628c6a0 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e2c0 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628de20 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628df50 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e080 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e1b0 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e8b0 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e410 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e540 2024-02-10T08:10:29.601 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e670 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e7a0 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ee60 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628e9c0 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628eb10 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ec20 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ed50 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f430 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628ef90 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f0a0 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f1b0 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f2c0 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628fa20 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f580 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f690 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f7e0 2024-02-10T08:10:29.602 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628fb50 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290000 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290130 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290700 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628fca0 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628fdb0 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628fee0 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628f910 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290260 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290390 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262904c0 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262905d0 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290cf0 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290850 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290960 2024-02-10T08:10:29.603 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290ab0 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290e20 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262912b0 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262913e0 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d880 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628d9b0 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262919b0 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291510 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291660 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291770 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262918a0 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291f80 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291ae0 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291c10 2024-02-10T08:10:29.604 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291d60 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262920b0 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292540 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292690 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26290f30 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26291060 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2628dac0 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262927c0 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292c70 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292d80 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293370 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292910 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292a40 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292b50 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262934a0 2024-02-10T08:10:29.605 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292ed0 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292fe0 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293130 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262935f0 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293a80 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293bb0 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262921e0 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292310 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26292420 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293ce0 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262941c0 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294340 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294930 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293e10 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293f40 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294490 2024-02-10T08:10:29.606 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262945c0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262946f0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294820 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294050 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294a60 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294bd0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262951c0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294d20 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294e70 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26294fc0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262952f0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262957a0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262958b0 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293720 2024-02-10T08:10:29.607 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293830 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26293960 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262959e0 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26295e70 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26295fa0 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26295440 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26295570 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296590 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262960f0 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296200 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26295680 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296330 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296460 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262966e0 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296870 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296e60 2024-02-10T08:10:29.608 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262969c0 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296af0 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296c00 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296d30 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297410 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26296f70 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297080 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262971b0 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262972e0 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297a00 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297560 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297690 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262977c0 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262978f0 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297fd0 2024-02-10T08:10:29.609 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297b30 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297c80 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26297db0 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298100 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262985e0 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298760 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26295b10 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26295c40 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298d30 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298890 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298230 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262996b0 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299210 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298360 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298490 2024-02-10T08:10:29.610 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299340 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299470 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262997e0 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299c90 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299dc0 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298e60 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26298f90 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262995a0 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262990c0 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629a390 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299ef0 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629a040 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629a190 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629a4c0 2024-02-10T08:10:29.611 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629a950 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629aa60 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299910 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299a40 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b050 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629abb0 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629ace0 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629ae10 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629af40 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b620 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b180 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b2d0 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b400 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b750 2024-02-10T08:10:29.612 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629bbe0 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629bd10 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629a5f0 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629a720 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c2e0 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629be40 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629bf70 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c0a0 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c1d0 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c8b0 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c410 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c540 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c670 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c7a0 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629ce80 2024-02-10T08:10:29.613 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629c9e0 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629cb30 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629cc40 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629cd70 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629d450 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629cfb0 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629d100 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629d230 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629d580 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629da10 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629db40 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629d340 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629dc70 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629dde0 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b26299b50 2024-02-10T08:10:29.614 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629df30 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e080 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b880 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629b9d0 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e670 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e1d0 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e300 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e430 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e560 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629ec40 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e7a0 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629e8d0 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629ea00 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629eb30 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629f210 2024-02-10T08:10:29.615 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629ed70 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629eea0 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629eff0 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629f340 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629f7d0 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629f900 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629f100 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629fa30 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629fb80 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629f470 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629f5c0 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0150 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629fcb0 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629fde0 2024-02-10T08:10:29.616 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629ff10 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0040 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0720 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0280 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a03d0 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0500 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0850 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0d30 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0eb0 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629d6b0 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b2629d7e0 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1480 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0fe0 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1110 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1240 2024-02-10T08:10:29.617 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1370 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1a50 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a15b0 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a16e0 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1830 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1ba0 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2080 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2200 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0980 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a0ab0 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a27d0 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2330 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2460 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2590 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a26c0 2024-02-10T08:10:29.618 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2da0 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2900 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2a30 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2b80 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a2ed0 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3360 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3490 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1cd0 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1e20 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3a40 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a1f50 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3b70 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a35a0 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a36d0 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3800 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3930 2024-02-10T08:10:29.619 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4140 2024-02-10T08:10:29.620 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3ca0 2024-02-10T08:10:29.620 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3df0 2024-02-10T08:10:29.620 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3f20 2024-02-10T08:10:29.620 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4270 2024-02-10T08:10:29.620 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4700 2024-02-10T08:10:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:29 smithi070 ceph-mon[29548]: pgmap v1639: 292 pgs: 32 unknown, 260 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:10:30.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:29 smithi070 ceph-mon[39923]: pgmap v1639: 292 pgs: 32 unknown, 260 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:10:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:29 smithi155 ceph-mon[32688]: pgmap v1639: 292 pgs: 32 unknown, 260 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:10:30.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:10:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:10:31.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:30 smithi070 ceph-mon[29548]: osdmap e756: 8 total, 8 up, 8 in 2024-02-10T08:10:31.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:30 smithi070 ceph-mon[39923]: osdmap e756: 8 total, 8 up, 8 in 2024-02-10T08:10:31.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:30 smithi155 ceph-mon[32688]: osdmap e756: 8 total, 8 up, 8 in 2024-02-10T08:10:32.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:32.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[29548]: pgmap v1642: 260 pgs: 32 unknown, 3 creating+activating, 225 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:32.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[29548]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:32.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[29548]: osdmap e757: 8 total, 8 up, 8 in 2024-02-10T08:10:32.061 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:32.061 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[39923]: pgmap v1642: 260 pgs: 32 unknown, 3 creating+activating, 225 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:32.061 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[39923]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:32.061 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:31 smithi070 ceph-mon[39923]: osdmap e757: 8 total, 8 up, 8 in 2024-02-10T08:10:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:31 smithi155 ceph-mon[32688]: pgmap v1642: 260 pgs: 32 unknown, 3 creating+activating, 225 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:31 smithi155 ceph-mon[32688]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:31 smithi155 ceph-mon[32688]: osdmap e757: 8 total, 8 up, 8 in 2024-02-10T08:10:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:10:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:10:33.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[29548]: osdmap e758: 8 total, 8 up, 8 in 2024-02-10T08:10:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:33.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[29548]: pgmap v1645: 260 pgs: 64 unknown, 3 creating+activating, 193 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[39923]: osdmap e758: 8 total, 8 up, 8 in 2024-02-10T08:10:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:33.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:32 smithi070 ceph-mon[39923]: pgmap v1645: 260 pgs: 64 unknown, 3 creating+activating, 193 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:32 smithi155 ceph-mon[32688]: osdmap e758: 8 total, 8 up, 8 in 2024-02-10T08:10:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:32 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:32 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:33.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:32 smithi155 ceph-mon[32688]: pgmap v1645: 260 pgs: 64 unknown, 3 creating+activating, 193 active+clean; 582 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262 watch_notify: Running main() from gmock_main.cc 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [----------] Global test environment set-up. 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: handle_notify cookie 93978194561008 notify_id 3229815406595 notifier_gid 24913 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (1808459 ms) 2024-02-10T08:10:33.814 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: Trying... 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: handle_notify cookie 93978206440272 notify_id 3242700308487 notifier_gid 36347 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: Waiting for 3.000000000s 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: Timed out. 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: Flushing... 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: Flushed... 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7053 ms) 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (1815512 ms total) 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [----------] Global test environment tear-down 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (1815512 ms total) 2024-02-10T08:10:33.815 INFO:tasks.workunit.client.0.smithi070.stdout: watch_notify: [ PASSED ] 2 tests. 2024-02-10T08:10:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[29548]: osdmap e759: 8 total, 8 up, 8 in 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[39923]: osdmap e759: 8 total, 8 up, 8 in 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:33 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:33 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:33 smithi155 ceph-mon[32688]: osdmap e759: 8 total, 8 up, 8 in 2024-02-10T08:10:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T08:10:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:34.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:33 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:35.244 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:35.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[39923]: osdmap e760: 8 total, 8 up, 8 in 2024-02-10T08:10:35.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2"}]: dispatch 2024-02-10T08:10:35.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:35.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:35.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[39923]: pgmap v1648: 196 pgs: 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:10:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[29548]: osdmap e760: 8 total, 8 up, 8 in 2024-02-10T08:10:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2"}]: dispatch 2024-02-10T08:10:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:35.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:34 smithi070 ceph-mon[29548]: pgmap v1648: 196 pgs: 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:10:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:34 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:34 smithi155 ceph-mon[32688]: osdmap e760: 8 total, 8 up, 8 in 2024-02-10T08:10:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2"}]: dispatch 2024-02-10T08:10:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:34 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:10:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:34 smithi155 ceph-mon[32688]: pgmap v1648: 196 pgs: 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:10:36.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:36.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[39923]: osdmap e761: 8 total, 8 up, 8 in 2024-02-10T08:10:36.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T08:10:36.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:10:36.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:10:36.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:36.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[29548]: osdmap e761: 8 total, 8 up, 8 in 2024-02-10T08:10:36.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T08:10:36.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:10:36.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:35 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:10:36.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:35 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi070-77493-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:10:36.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:35 smithi155 ceph-mon[32688]: osdmap e761: 8 total, 8 up, 8 in 2024-02-10T08:10:36.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T08:10:36.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:35 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:10:36.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:35 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:10:37.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:10:37.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[29548]: osdmap e762: 8 total, 8 up, 8 in 2024-02-10T08:10:37.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:10:37.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:10:37.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[29548]: pgmap v1651: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:10:37.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:10:37.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[39923]: osdmap e762: 8 total, 8 up, 8 in 2024-02-10T08:10:37.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:10:37.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:10:37.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:36 smithi070 ceph-mon[39923]: pgmap v1651: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:10:37.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:36 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:10:37.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:36 smithi155 ceph-mon[32688]: osdmap e762: 8 total, 8 up, 8 in 2024-02-10T08:10:37.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:36 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:10:37.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:36 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:10:37.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:36 smithi155 ceph-mon[32688]: pgmap v1651: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout:a4830 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3000 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a3130 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4e00 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4960 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4a90 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4bc0 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4cf0 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a53d0 2024-02-10T08:10:37.854 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a4f30 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5060 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5190 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a52c0 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a59a0 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5500 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5630 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5780 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5ad0 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5f60 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6090 2024-02-10T08:10:37.855 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5890 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a61c0 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6310 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5c00 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5d10 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6900 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6460 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6590 2024-02-10T08:10:37.856 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a5e20 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a66c0 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6ed0 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6a30 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6b80 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a6cd0 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7000 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7490 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a75c0 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7b90 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7130 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7260 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a76f0 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7820 2024-02-10T08:10:37.857 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7950 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7a80 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8180 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7ce0 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7e10 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a7f60 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a82b0 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8740 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8870 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a43a0 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a44d0 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8e40 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a89a0 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8ad0 2024-02-10T08:10:37.858 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8c00 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8d30 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a9410 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a8f70 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a90a0 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a91f0 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a9540 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a99d0 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: waiting 0x564b262a9b00 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: done. 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1810778 ms) 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1810778 ms total) 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: 2024-02-10T08:10:37.859 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.Exec (26 ms) 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (2 ms) 2024-02-10T08:10:37.860 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-10T08:10:38.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[39923]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[39923]: osdmap e763: 8 total, 8 up, 8 in 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[29548]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[29548]: osdmap e763: 8 total, 8 up, 8 in 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:10:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:37 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:10:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:37 smithi155 ceph-mon[32688]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:37 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:10:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:37 smithi155 ceph-mon[32688]: osdmap e763: 8 total, 8 up, 8 in 2024-02-10T08:10:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:37 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2952521729' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:10:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:37 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:10:39.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:38 smithi155 ceph-mon[32688]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:10:39.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:38 smithi155 ceph-mon[32688]: osdmap e764: 8 total, 8 up, 8 in 2024-02-10T08:10:39.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:38 smithi155 ceph-mon[32688]: pgmap v1654: 196 pgs: 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:39.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:38 smithi070 ceph-mon[39923]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:10:39.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:38 smithi070 ceph-mon[39923]: osdmap e764: 8 total, 8 up, 8 in 2024-02-10T08:10:39.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:38 smithi070 ceph-mon[39923]: pgmap v1654: 196 pgs: 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:39.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:38 smithi070 ceph-mon[29548]: from='client.26323 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi070-77493-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:10:39.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:38 smithi070 ceph-mon[29548]: osdmap e764: 8 total, 8 up, 8 in 2024-02-10T08:10:39.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:38 smithi070 ceph-mon[29548]: pgmap v1654: 196 pgs: 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:40.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:39 smithi070 ceph-mon[39923]: osdmap e765: 8 total, 8 up, 8 in 2024-02-10T08:10:40.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:39 smithi070 ceph-mon[29548]: osdmap e765: 8 total, 8 up, 8 in 2024-02-10T08:10:40.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:39 smithi155 ceph-mon[32688]: osdmap e765: 8 total, 8 up, 8 in 2024-02-10T08:10:40.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:10:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[29548]: osdmap e766: 8 total, 8 up, 8 in 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[29548]: pgmap v1657: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[29548]: osdmap e767: 8 total, 8 up, 8 in 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[39923]: osdmap e766: 8 total, 8 up, 8 in 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[39923]: pgmap v1657: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:40 smithi070 ceph-mon[39923]: osdmap e767: 8 total, 8 up, 8 in 2024-02-10T08:10:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:40 smithi155 ceph-mon[32688]: osdmap e766: 8 total, 8 up, 8 in 2024-02-10T08:10:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:40 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:40 smithi155 ceph-mon[32688]: pgmap v1657: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:40 smithi155 ceph-mon[32688]: osdmap e767: 8 total, 8 up, 8 in 2024-02-10T08:10:42.337 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:10:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:10:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:10:42.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:42 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:42.742 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:42 smithi070 ceph-mon[29548]: osdmap e768: 8 total, 8 up, 8 in 2024-02-10T08:10:42.742 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:42 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:42.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:42 smithi070 ceph-mon[39923]: osdmap e768: 8 total, 8 up, 8 in 2024-02-10T08:10:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:42 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:42.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:42 smithi155 ceph-mon[32688]: osdmap e768: 8 total, 8 up, 8 in 2024-02-10T08:10:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:43 smithi070 ceph-mon[29548]: pgmap v1660: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:43.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:43 smithi070 ceph-mon[29548]: osdmap e769: 8 total, 8 up, 8 in 2024-02-10T08:10:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:43 smithi070 ceph-mon[39923]: pgmap v1660: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:43.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:43 smithi070 ceph-mon[39923]: osdmap e769: 8 total, 8 up, 8 in 2024-02-10T08:10:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:43 smithi155 ceph-mon[32688]: pgmap v1660: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 824 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:44.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:43 smithi155 ceph-mon[32688]: osdmap e769: 8 total, 8 up, 8 in 2024-02-10T08:10:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:44 smithi070 ceph-mon[29548]: osdmap e770: 8 total, 8 up, 8 in 2024-02-10T08:10:45.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:44 smithi070 ceph-mon[29548]: pgmap v1663: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:44 smithi070 ceph-mon[39923]: osdmap e770: 8 total, 8 up, 8 in 2024-02-10T08:10:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:44 smithi070 ceph-mon[39923]: pgmap v1663: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:44 smithi155 ceph-mon[32688]: osdmap e770: 8 total, 8 up, 8 in 2024-02-10T08:10:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:44 smithi155 ceph-mon[32688]: pgmap v1663: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:46.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:45 smithi070 ceph-mon[29548]: osdmap e771: 8 total, 8 up, 8 in 2024-02-10T08:10:46.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:45 smithi070 ceph-mon[39923]: osdmap e771: 8 total, 8 up, 8 in 2024-02-10T08:10:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:45 smithi155 ceph-mon[32688]: osdmap e771: 8 total, 8 up, 8 in 2024-02-10T08:10:47.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:46 smithi070 ceph-mon[29548]: osdmap e772: 8 total, 8 up, 8 in 2024-02-10T08:10:47.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:46 smithi070 ceph-mon[29548]: pgmap v1666: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:47.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:46 smithi070 ceph-mon[39923]: osdmap e772: 8 total, 8 up, 8 in 2024-02-10T08:10:47.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:46 smithi070 ceph-mon[39923]: pgmap v1666: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:46 smithi155 ceph-mon[32688]: osdmap e772: 8 total, 8 up, 8 in 2024-02-10T08:10:47.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:46 smithi155 ceph-mon[32688]: pgmap v1666: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:47 smithi070 ceph-mon[29548]: osdmap e773: 8 total, 8 up, 8 in 2024-02-10T08:10:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:47 smithi070 ceph-mon[39923]: osdmap e773: 8 total, 8 up, 8 in 2024-02-10T08:10:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:47 smithi155 ceph-mon[32688]: osdmap e773: 8 total, 8 up, 8 in 2024-02-10T08:10:49.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:48 smithi070 ceph-mon[39923]: osdmap e774: 8 total, 8 up, 8 in 2024-02-10T08:10:49.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:48 smithi070 ceph-mon[39923]: pgmap v1669: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:49.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:48 smithi070 ceph-mon[39923]: osdmap e775: 8 total, 8 up, 8 in 2024-02-10T08:10:49.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:48 smithi070 ceph-mon[29548]: osdmap e774: 8 total, 8 up, 8 in 2024-02-10T08:10:49.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:48 smithi070 ceph-mon[29548]: pgmap v1669: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:49.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:48 smithi070 ceph-mon[29548]: osdmap e775: 8 total, 8 up, 8 in 2024-02-10T08:10:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:48 smithi155 ceph-mon[32688]: osdmap e774: 8 total, 8 up, 8 in 2024-02-10T08:10:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:48 smithi155 ceph-mon[32688]: pgmap v1669: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 834 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:49.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:48 smithi155 ceph-mon[32688]: osdmap e775: 8 total, 8 up, 8 in 2024-02-10T08:10:50.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:10:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:10:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:50 smithi070 ceph-mon[29548]: osdmap e776: 8 total, 8 up, 8 in 2024-02-10T08:10:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:50 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:50 smithi070 ceph-mon[29548]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 847 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:50 smithi070 ceph-mon[39923]: osdmap e776: 8 total, 8 up, 8 in 2024-02-10T08:10:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:50 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:50 smithi070 ceph-mon[39923]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 847 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:50 smithi155 ceph-mon[32688]: osdmap e776: 8 total, 8 up, 8 in 2024-02-10T08:10:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:50 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:10:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:50 smithi155 ceph-mon[32688]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 847 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:52.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:10:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:10:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:10:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:51 smithi070 ceph-mon[39923]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:51 smithi070 ceph-mon[39923]: osdmap e777: 8 total, 8 up, 8 in 2024-02-10T08:10:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:51 smithi070 ceph-mon[29548]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:52.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:51 smithi070 ceph-mon[29548]: osdmap e777: 8 total, 8 up, 8 in 2024-02-10T08:10:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:51 smithi155 ceph-mon[32688]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:51 smithi155 ceph-mon[32688]: osdmap e777: 8 total, 8 up, 8 in 2024-02-10T08:10:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:52 smithi070 ceph-mon[29548]: osdmap e778: 8 total, 8 up, 8 in 2024-02-10T08:10:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:52 smithi070 ceph-mon[29548]: pgmap v1675: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 847 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:52 smithi070 ceph-mon[39923]: osdmap e778: 8 total, 8 up, 8 in 2024-02-10T08:10:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:52 smithi070 ceph-mon[39923]: pgmap v1675: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 847 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:52 smithi155 ceph-mon[32688]: osdmap e778: 8 total, 8 up, 8 in 2024-02-10T08:10:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:52 smithi155 ceph-mon[32688]: pgmap v1675: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 847 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:54.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:53 smithi070 ceph-mon[29548]: osdmap e779: 8 total, 8 up, 8 in 2024-02-10T08:10:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:53 smithi070 ceph-mon[39923]: osdmap e779: 8 total, 8 up, 8 in 2024-02-10T08:10:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:53 smithi155 ceph-mon[32688]: osdmap e779: 8 total, 8 up, 8 in 2024-02-10T08:10:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:54 smithi070 ceph-mon[29548]: osdmap e780: 8 total, 8 up, 8 in 2024-02-10T08:10:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:54 smithi070 ceph-mon[29548]: pgmap v1678: 228 pgs: 228 active+clean; 583 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:10:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:54 smithi070 ceph-mon[39923]: osdmap e780: 8 total, 8 up, 8 in 2024-02-10T08:10:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:54 smithi070 ceph-mon[39923]: pgmap v1678: 228 pgs: 228 active+clean; 583 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:10:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:54 smithi155 ceph-mon[32688]: osdmap e780: 8 total, 8 up, 8 in 2024-02-10T08:10:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:54 smithi155 ceph-mon[32688]: pgmap v1678: 228 pgs: 228 active+clean; 583 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:10:56.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:55 smithi070 ceph-mon[29548]: osdmap e781: 8 total, 8 up, 8 in 2024-02-10T08:10:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:55 smithi070 ceph-mon[39923]: osdmap e781: 8 total, 8 up, 8 in 2024-02-10T08:10:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:55 smithi155 ceph-mon[32688]: osdmap e781: 8 total, 8 up, 8 in 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5014 ms) 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (18196 ms) 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] 8 tests from LibRadosMisc (23242 ms total) 2024-02-10T08:10:57.011 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: 2024-02-10T08:10:57.012 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [----------] Global test environment tear-down 2024-02-10T08:10:57.012 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1839046 ms total) 2024-02-10T08:10:57.012 INFO:tasks.workunit.client.0.smithi070.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77714 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77714 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78208 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78208 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78221 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78221 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.015 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77354 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77354 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78070 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78070 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77925 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77925 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77213 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77213 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77275 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77275 2024-02-10T08:10:57.016 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77978 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77978 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77204 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77204 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77225 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77225 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78372 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78372 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77536 2024-02-10T08:10:57.017 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77536 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78444 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78444 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77660 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77660 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78160 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78160 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78513 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78513 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77560 2024-02-10T08:10:57.018 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77560 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78013 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78013 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78351 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78351 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78124 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78124 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78407 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78407 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.019 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77893 2024-02-10T08:10:57.020 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77893 2024-02-10T08:10:57.020 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.020 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77467 2024-02-10T08:10:57.020 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77467 2024-02-10T08:10:57.020 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:10:57.020 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78475 2024-02-10T08:10:57.020 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78475 2024-02-10T08:10:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:56 smithi070 ceph-mon[29548]: osdmap e782: 8 total, 8 up, 8 in 2024-02-10T08:10:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:56 smithi070 ceph-mon[29548]: pgmap v1681: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:57.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:56 smithi070 ceph-mon[39923]: osdmap e782: 8 total, 8 up, 8 in 2024-02-10T08:10:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:56 smithi070 ceph-mon[39923]: pgmap v1681: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:56 smithi155 ceph-mon[32688]: osdmap e782: 8 total, 8 up, 8 in 2024-02-10T08:10:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:56 smithi155 ceph-mon[32688]: pgmap v1681: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 865 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:10:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:10:58.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:57 smithi070 ceph-mon[29548]: osdmap e783: 8 total, 8 up, 8 in 2024-02-10T08:10:58.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:57 smithi070 ceph-mon[29548]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:57 smithi070 ceph-mon[39923]: osdmap e783: 8 total, 8 up, 8 in 2024-02-10T08:10:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:57 smithi070 ceph-mon[39923]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:57 smithi155 ceph-mon[32688]: osdmap e783: 8 total, 8 up, 8 in 2024-02-10T08:10:58.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:57 smithi155 ceph-mon[32688]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:10:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:58 smithi155 ceph-mon[32688]: osdmap e784: 8 total, 8 up, 8 in 2024-02-10T08:10:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:10:58 smithi155 ceph-mon[32688]: pgmap v1684: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:59.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:58 smithi070 ceph-mon[39923]: osdmap e784: 8 total, 8 up, 8 in 2024-02-10T08:10:59.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:10:58 smithi070 ceph-mon[39923]: pgmap v1684: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:10:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:58 smithi070 ceph-mon[29548]: osdmap e784: 8 total, 8 up, 8 in 2024-02-10T08:10:59.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:10:58 smithi070 ceph-mon[29548]: pgmap v1684: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:00.424 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:00 smithi155 ceph-mon[32688]: osdmap e785: 8 total, 8 up, 8 in 2024-02-10T08:11:00.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:00 smithi070 ceph-mon[29548]: osdmap e785: 8 total, 8 up, 8 in 2024-02-10T08:11:00.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:00 smithi070 ceph-mon[39923]: osdmap e785: 8 total, 8 up, 8 in 2024-02-10T08:11:00.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:11:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:11:01.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[39923]: osdmap e786: 8 total, 8 up, 8 in 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[39923]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[29548]: osdmap e786: 8 total, 8 up, 8 in 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:01.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[29548]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-02-10T08:11:01.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:11:01.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:11:01.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:01 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:11:01.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:01 smithi155 ceph-mon[32688]: osdmap e786: 8 total, 8 up, 8 in 2024-02-10T08:11:01.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:01 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:11:01.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:01.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:01 smithi155 ceph-mon[32688]: pgmap v1687: 196 pgs: 196 active+clean; 583 KiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-02-10T08:11:01.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:01 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:11:01.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:01 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:11:01.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:01 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:11:02.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:02 smithi070 ceph-mon[29548]: osdmap e787: 8 total, 8 up, 8 in 2024-02-10T08:11:02.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:02 smithi070 ceph-mon[39923]: osdmap e787: 8 total, 8 up, 8 in 2024-02-10T08:11:02.492 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:11:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:11:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:11:02.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:02 smithi155 ceph-mon[32688]: osdmap e787: 8 total, 8 up, 8 in 2024-02-10T08:11:03.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:03 smithi070 ceph-mon[29548]: osdmap e788: 8 total, 8 up, 8 in 2024-02-10T08:11:03.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:03 smithi070 ceph-mon[29548]: pgmap v1690: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:03.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:03 smithi070 ceph-mon[39923]: osdmap e788: 8 total, 8 up, 8 in 2024-02-10T08:11:03.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:03 smithi070 ceph-mon[39923]: pgmap v1690: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:03.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:03 smithi155 ceph-mon[32688]: osdmap e788: 8 total, 8 up, 8 in 2024-02-10T08:11:03.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:03 smithi155 ceph-mon[32688]: pgmap v1690: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 866 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:04.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:04 smithi070 ceph-mon[39923]: osdmap e789: 8 total, 8 up, 8 in 2024-02-10T08:11:04.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:04 smithi070 ceph-mon[29548]: osdmap e789: 8 total, 8 up, 8 in 2024-02-10T08:11:04.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:04 smithi155 ceph-mon[32688]: osdmap e789: 8 total, 8 up, 8 in 2024-02-10T08:11:05.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:05 smithi070 ceph-mon[29548]: osdmap e790: 8 total, 8 up, 8 in 2024-02-10T08:11:05.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:05 smithi070 ceph-mon[29548]: pgmap v1693: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:05 smithi070 ceph-mon[39923]: osdmap e790: 8 total, 8 up, 8 in 2024-02-10T08:11:05.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:05 smithi070 ceph-mon[39923]: pgmap v1693: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:05.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:05 smithi155 ceph-mon[32688]: osdmap e790: 8 total, 8 up, 8 in 2024-02-10T08:11:05.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:05 smithi155 ceph-mon[32688]: pgmap v1693: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:06.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:06 smithi070 ceph-mon[29548]: osdmap e791: 8 total, 8 up, 8 in 2024-02-10T08:11:06.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:06 smithi070 ceph-mon[39923]: osdmap e791: 8 total, 8 up, 8 in 2024-02-10T08:11:06.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:06 smithi155 ceph-mon[32688]: osdmap e791: 8 total, 8 up, 8 in 2024-02-10T08:11:07.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:07 smithi070 ceph-mon[29548]: osdmap e792: 8 total, 8 up, 8 in 2024-02-10T08:11:07.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:07 smithi070 ceph-mon[29548]: pgmap v1696: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:11:07.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:07 smithi070 ceph-mon[39923]: osdmap e792: 8 total, 8 up, 8 in 2024-02-10T08:11:07.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:07 smithi070 ceph-mon[39923]: pgmap v1696: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:11:07.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:07 smithi155 ceph-mon[32688]: osdmap e792: 8 total, 8 up, 8 in 2024-02-10T08:11:07.507 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:07 smithi155 ceph-mon[32688]: pgmap v1696: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:11:08.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:08 smithi070 ceph-mon[29548]: osdmap e793: 8 total, 8 up, 8 in 2024-02-10T08:11:08.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:08 smithi070 ceph-mon[29548]: osdmap e794: 8 total, 8 up, 8 in 2024-02-10T08:11:08.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:08 smithi070 ceph-mon[39923]: osdmap e793: 8 total, 8 up, 8 in 2024-02-10T08:11:08.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:08 smithi070 ceph-mon[39923]: osdmap e794: 8 total, 8 up, 8 in 2024-02-10T08:11:08.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:08 smithi155 ceph-mon[32688]: osdmap e793: 8 total, 8 up, 8 in 2024-02-10T08:11:08.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:08 smithi155 ceph-mon[32688]: osdmap e794: 8 total, 8 up, 8 in 2024-02-10T08:11:09.407 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:09 smithi155 ceph-mon[32688]: pgmap v1699: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:09.407 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:09 smithi155 ceph-mon[32688]: osdmap e795: 8 total, 8 up, 8 in 2024-02-10T08:11:09.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:09 smithi070 ceph-mon[29548]: pgmap v1699: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:09.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:09 smithi070 ceph-mon[29548]: osdmap e795: 8 total, 8 up, 8 in 2024-02-10T08:11:09.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:09 smithi070 ceph-mon[39923]: pgmap v1699: 196 pgs: 196 active+clean; 583 KiB data, 884 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:09.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:09 smithi070 ceph-mon[39923]: osdmap e795: 8 total, 8 up, 8 in 2024-02-10T08:11:10.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:11:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:11:11.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:11 smithi070 ceph-mon[29548]: osdmap e796: 8 total, 8 up, 8 in 2024-02-10T08:11:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:11.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:11 smithi070 ceph-mon[29548]: pgmap v1702: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:11:11.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:11 smithi070 ceph-mon[39923]: osdmap e796: 8 total, 8 up, 8 in 2024-02-10T08:11:11.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:11.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:11 smithi070 ceph-mon[39923]: pgmap v1702: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:11:11.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:11 smithi155 ceph-mon[32688]: osdmap e796: 8 total, 8 up, 8 in 2024-02-10T08:11:11.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:11.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:11 smithi155 ceph-mon[32688]: pgmap v1702: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:11:12.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:12 smithi070 ceph-mon[29548]: osdmap e797: 8 total, 8 up, 8 in 2024-02-10T08:11:12.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:12 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:12.492 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:11:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:11:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:11:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:12 smithi070 ceph-mon[39923]: osdmap e797: 8 total, 8 up, 8 in 2024-02-10T08:11:12.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:12 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:12.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:12 smithi155 ceph-mon[32688]: osdmap e797: 8 total, 8 up, 8 in 2024-02-10T08:11:12.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:12 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:13.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:13 smithi070 ceph-mon[29548]: osdmap e798: 8 total, 8 up, 8 in 2024-02-10T08:11:13.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:13 smithi070 ceph-mon[29548]: pgmap v1705: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:11:13.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:13 smithi070 ceph-mon[39923]: osdmap e798: 8 total, 8 up, 8 in 2024-02-10T08:11:13.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:13 smithi070 ceph-mon[39923]: pgmap v1705: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:11:13.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:13 smithi155 ceph-mon[32688]: osdmap e798: 8 total, 8 up, 8 in 2024-02-10T08:11:13.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:13 smithi155 ceph-mon[32688]: pgmap v1705: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 884 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:11:14.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:14 smithi070 ceph-mon[29548]: osdmap e799: 8 total, 8 up, 8 in 2024-02-10T08:11:14.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:14 smithi070 ceph-mon[39923]: osdmap e799: 8 total, 8 up, 8 in 2024-02-10T08:11:14.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:14 smithi155 ceph-mon[32688]: osdmap e799: 8 total, 8 up, 8 in 2024-02-10T08:11:15.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:15 smithi070 ceph-mon[29548]: osdmap e800: 8 total, 8 up, 8 in 2024-02-10T08:11:15.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:15 smithi070 ceph-mon[29548]: pgmap v1708: 196 pgs: 196 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 256 KiB/s wr, 1 op/s 2024-02-10T08:11:15.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:15 smithi070 ceph-mon[39923]: osdmap e800: 8 total, 8 up, 8 in 2024-02-10T08:11:15.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:15 smithi070 ceph-mon[39923]: pgmap v1708: 196 pgs: 196 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 256 KiB/s wr, 1 op/s 2024-02-10T08:11:15.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:15 smithi155 ceph-mon[32688]: osdmap e800: 8 total, 8 up, 8 in 2024-02-10T08:11:15.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:15 smithi155 ceph-mon[32688]: pgmap v1708: 196 pgs: 196 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 256 KiB/s wr, 1 op/s 2024-02-10T08:11:16.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:16 smithi070 ceph-mon[29548]: osdmap e801: 8 total, 8 up, 8 in 2024-02-10T08:11:16.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:16 smithi070 ceph-mon[39923]: osdmap e801: 8 total, 8 up, 8 in 2024-02-10T08:11:16.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:16 smithi155 ceph-mon[32688]: osdmap e801: 8 total, 8 up, 8 in 2024-02-10T08:11:17.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:17 smithi070 ceph-mon[29548]: osdmap e802: 8 total, 8 up, 8 in 2024-02-10T08:11:17.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:17 smithi070 ceph-mon[29548]: pgmap v1711: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:17.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:17 smithi070 ceph-mon[39923]: osdmap e802: 8 total, 8 up, 8 in 2024-02-10T08:11:17.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:17 smithi070 ceph-mon[39923]: pgmap v1711: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:17.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:17 smithi155 ceph-mon[32688]: osdmap e802: 8 total, 8 up, 8 in 2024-02-10T08:11:17.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:17 smithi155 ceph-mon[32688]: pgmap v1711: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:18.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:18 smithi070 ceph-mon[29548]: osdmap e803: 8 total, 8 up, 8 in 2024-02-10T08:11:18.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:18 smithi070 ceph-mon[39923]: osdmap e803: 8 total, 8 up, 8 in 2024-02-10T08:11:18.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:18 smithi155 ceph-mon[32688]: osdmap e803: 8 total, 8 up, 8 in 2024-02-10T08:11:19.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:19 smithi070 ceph-mon[29548]: osdmap e804: 8 total, 8 up, 8 in 2024-02-10T08:11:19.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:19 smithi070 ceph-mon[29548]: pgmap v1714: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:19.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:19 smithi070 ceph-mon[39923]: osdmap e804: 8 total, 8 up, 8 in 2024-02-10T08:11:19.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:19 smithi070 ceph-mon[39923]: pgmap v1714: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:19.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:19 smithi155 ceph-mon[32688]: osdmap e804: 8 total, 8 up, 8 in 2024-02-10T08:11:19.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:19 smithi155 ceph-mon[32688]: pgmap v1714: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 894 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:11:20.443 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:20 smithi155 ceph-mon[32688]: osdmap e805: 8 total, 8 up, 8 in 2024-02-10T08:11:20.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:20 smithi070 ceph-mon[29548]: osdmap e805: 8 total, 8 up, 8 in 2024-02-10T08:11:20.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:20 smithi070 ceph-mon[39923]: osdmap e805: 8 total, 8 up, 8 in 2024-02-10T08:11:20.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:11:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:11:21.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:21 smithi070 ceph-mon[39923]: osdmap e806: 8 total, 8 up, 8 in 2024-02-10T08:11:21.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:21.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:21 smithi070 ceph-mon[39923]: pgmap v1717: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:21.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:21 smithi070 ceph-mon[29548]: osdmap e806: 8 total, 8 up, 8 in 2024-02-10T08:11:21.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:21.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:21 smithi070 ceph-mon[29548]: pgmap v1717: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:21.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:21 smithi155 ceph-mon[32688]: osdmap e806: 8 total, 8 up, 8 in 2024-02-10T08:11:21.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:21.506 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:21 smithi155 ceph-mon[32688]: pgmap v1717: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:22.393 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:22 smithi070 ceph-mon[29548]: osdmap e807: 8 total, 8 up, 8 in 2024-02-10T08:11:22.393 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:22 smithi070 ceph-mon[39923]: osdmap e807: 8 total, 8 up, 8 in 2024-02-10T08:11:22.394 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:11:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:11:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:11:22.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:22 smithi155 ceph-mon[32688]: osdmap e807: 8 total, 8 up, 8 in 2024-02-10T08:11:23.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:23 smithi070 ceph-mon[29548]: osdmap e808: 8 total, 8 up, 8 in 2024-02-10T08:11:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:23 smithi070 ceph-mon[29548]: pgmap v1720: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:23.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:23 smithi070 ceph-mon[39923]: osdmap e808: 8 total, 8 up, 8 in 2024-02-10T08:11:23.492 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:23 smithi070 ceph-mon[39923]: pgmap v1720: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:23.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:23 smithi155 ceph-mon[32688]: osdmap e808: 8 total, 8 up, 8 in 2024-02-10T08:11:23.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:23 smithi155 ceph-mon[32688]: pgmap v1720: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:24.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:24 smithi070 ceph-mon[39923]: osdmap e809: 8 total, 8 up, 8 in 2024-02-10T08:11:24.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:24 smithi070 ceph-mon[29548]: osdmap e809: 8 total, 8 up, 8 in 2024-02-10T08:11:24.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:24 smithi155 ceph-mon[32688]: osdmap e809: 8 total, 8 up, 8 in 2024-02-10T08:11:25.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:25 smithi070 ceph-mon[39923]: pgmap v1722: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 234 B/s wr, 1 op/s 2024-02-10T08:11:25.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:25 smithi070 ceph-mon[29548]: pgmap v1722: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 234 B/s wr, 1 op/s 2024-02-10T08:11:25.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:25 smithi155 ceph-mon[32688]: pgmap v1722: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 234 B/s wr, 1 op/s 2024-02-10T08:11:27.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:27 smithi070 ceph-mon[39923]: pgmap v1723: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-02-10T08:11:27.997 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:27 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:27.997 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:27 smithi070 ceph-mon[29548]: pgmap v1723: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-02-10T08:11:27.997 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:27 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:28.011 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:27 smithi155 ceph-mon[32688]: pgmap v1723: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 170 B/s wr, 0 op/s 2024-02-10T08:11:28.011 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:27 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:29.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:29 smithi070 ceph-mon[29548]: pgmap v1724: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 694 B/s rd, 138 B/s wr, 0 op/s 2024-02-10T08:11:29.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:29 smithi070 ceph-mon[39923]: pgmap v1724: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 694 B/s rd, 138 B/s wr, 0 op/s 2024-02-10T08:11:30.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:29 smithi155 ceph-mon[32688]: pgmap v1724: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 694 B/s rd, 138 B/s wr, 0 op/s 2024-02-10T08:11:30.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:11:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:11:31.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:31 smithi070 ceph-mon[29548]: pgmap v1725: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 122 B/s wr, 1 op/s 2024-02-10T08:11:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:31 smithi070 ceph-mon[39923]: pgmap v1725: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 122 B/s wr, 1 op/s 2024-02-10T08:11:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:32.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:31 smithi155 ceph-mon[32688]: pgmap v1725: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 122 B/s wr, 1 op/s 2024-02-10T08:11:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:11:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:11:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:11:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:33 smithi070 ceph-mon[29548]: pgmap v1726: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T08:11:33.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:33 smithi070 ceph-mon[29548]: osdmap e810: 8 total, 8 up, 8 in 2024-02-10T08:11:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:33 smithi070 ceph-mon[39923]: pgmap v1726: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T08:11:33.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:33 smithi070 ceph-mon[39923]: osdmap e810: 8 total, 8 up, 8 in 2024-02-10T08:11:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:33 smithi155 ceph-mon[32688]: pgmap v1726: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-10T08:11:34.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:33 smithi155 ceph-mon[32688]: osdmap e810: 8 total, 8 up, 8 in 2024-02-10T08:11:35.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:35 smithi070 ceph-mon[29548]: pgmap v1728: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:35.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:35 smithi070 ceph-mon[39923]: pgmap v1728: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:35 smithi155 ceph-mon[32688]: pgmap v1728: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:37.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:37 smithi070 ceph-mon[29548]: pgmap v1729: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:37.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:37 smithi070 ceph-mon[39923]: pgmap v1729: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:37 smithi155 ceph-mon[32688]: pgmap v1729: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:38.753 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: Running main() from gmock_main.cc 2024-02-10T08:11:38.753 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [----------] Global test environment set-up. 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (1810458 ms) 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5052 ms) 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (4040 ms) 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5023 ms) 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7031 ms) 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (1831604 ms total) 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: 2024-02-10T08:11:38.754 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5071 ms) 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6039 ms) 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8031 ms) 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6043 ms) 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4029 ms) 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi070-78656-11. 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: Waiting for snaps to purge. 2024-02-10T08:11:38.755 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19632 ms) 2024-02-10T08:11:38.756 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (48845 ms total) 2024-02-10T08:11:38.756 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: 2024-02-10T08:11:38.756 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [----------] Global test environment tear-down 2024-02-10T08:11:38.756 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (1880449 ms total) 2024-02-10T08:11:38.756 INFO:tasks.workunit.client.0.smithi070.stdout: snapshots: [ PASSED ] 11 tests. 2024-02-10T08:11:38.757 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.757 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77817 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77817 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77197 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77197 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77255 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77255 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77745 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77745 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77294 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77294 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.758 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77832 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77832 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=77846 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 77846 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78308 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78308 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=78567 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 78567 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ exit 0 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ cleanup 2024-02-10T08:11:38.759 INFO:tasks.workunit.client.0.smithi070.stderr:+ pkill -P 77192 2024-02-10T08:11:38.775 INFO:tasks.workunit.client.0.smithi070.stderr:+ true 2024-02-10T08:11:38.775 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-10T08:11:38.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-10T08:11:38.809 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-10T08:11:38.810 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-10T08:11:38.810 DEBUG:teuthology.orchestra.run.smithi070:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-10T08:11:38.869 INFO:tasks.workunit.client.0.smithi070.stderr:++ uuidgen 2024-02-10T08:11:38.880 INFO:tasks.workunit.client.0.smithi070.stderr:+ p=939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:11:38.880 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool create 939b67e9-3d3f-4cc4-aedd-d4506338adca 12 2024-02-10T08:11:38.963 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.958+0000 7f80d300e640 1 -- 172.21.15.70:0/962419862 >> v1:172.21.15.70:6790/0 conn(0x7f80cc10c9d0 legacy=0x7f80cc10ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:38.963 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.958+0000 7f80d300e640 1 -- 172.21.15.70:0/962419862 shutdown_connections 2024-02-10T08:11:38.963 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.958+0000 7f80d300e640 1 -- 172.21.15.70:0/962419862 >> 172.21.15.70:0/962419862 conn(0x7f80cc076c40 msgr2=0x7f80cc0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:11:38.963 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.958+0000 7f80d300e640 1 -- 172.21.15.70:0/962419862 shutdown_connections 2024-02-10T08:11:38.963 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.958+0000 7f80d300e640 1 -- 172.21.15.70:0/962419862 wait complete. 2024-02-10T08:11:38.963 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d300e640 1 Processor -- start 2024-02-10T08:11:38.963 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d300e640 1 -- start start 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d300e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80cc1a8120 con 0x7f80cc1001e0 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d300e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80cc1a9320 con 0x7f80cc108e60 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d300e640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f80cc1aa520 con 0x7f80cc10c9d0 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d0daa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f80cc1001e0 0x7f80cc105330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39726/0 (socket says 172.21.15.70:39726) 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d0daa640 1 -- 172.21.15.70:0/2341917311 learned_addr learned my addr 172.21.15.70:0/2341917311 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80d15ab640 1 --1- 172.21.15.70:0/2341917311 >> v1:172.21.15.70:6790/0 conn(0x7f80cc10c9d0 0x7f80cc1a6870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:59898/0 (socket says 172.21.15.70:59898) 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349505694 0 0) 0x7f80cc1aa520 con 0x7f80cc10c9d0 2024-02-10T08:11:38.964 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b8003520 con 0x7f80cc10c9d0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135728504 0 0) 0x7f80cc1a8120 con 0x7f80cc1001e0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80cc1aa520 con 0x7f80cc1001e0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519868824 0 0) 0x7f80cc1a9320 con 0x7f80cc108e60 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80cc1a8120 con 0x7f80cc108e60 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3749450719 0 0) 0x7f80b8003520 con 0x7f80cc10c9d0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80cc1a9320 con 0x7f80cc10c9d0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.959+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4052950826 0 0) 0x7f80cc1aa520 con 0x7f80cc1001e0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80b8003520 con 0x7f80cc1001e0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f80b40043a0 con 0x7f80cc10c9d0 2024-02-10T08:11:38.965 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f80c4001bb0 con 0x7f80cc1001e0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3833841307 0 0) 0x7f80cc1a9320 con 0x7f80cc10c9d0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 >> v1:172.21.15.155:6789/0 conn(0x7f80cc108e60 legacy=0x7f80cc1a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 >> v1:172.21.15.70:6789/0 conn(0x7f80cc1001e0 legacy=0x7f80cc105330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc1ab720 con 0x7f80cc10c9d0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc1aa710 con 0x7f80cc10c9d0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f80b4004660 con 0x7f80cc10c9d0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f80b4004e90 con 0x7f80cc10c9d0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.960+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc1aaba0 con 0x7f80cc10c9d0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.961+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc1016b0 con 0x7f80cc10c9d0 2024-02-10T08:11:38.966 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.961+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f80b4004ff0 con 0x7f80cc10c9d0 2024-02-10T08:11:38.967 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.962+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(811..811 src has 252..811) v4 ==== 7430+0+0 (unknown 1076365753 0 0) 0x7f80b4088fd0 con 0x7f80cc10c9d0 2024-02-10T08:11:38.968 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:38.967+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f80b4055e40 con 0x7f80cc10c9d0 2024-02-10T08:11:39.151 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.151+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12} v 0) v1 -- 0x7f80cc067f20 con 0x7f80cc10c9d0 2024-02-10T08:11:39.753 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.752+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]=0 pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' created v812) v1 ==== 176+0+0 (unknown 939370431 0 0) 0x7f80cc067f20 con 0x7f80cc10c9d0 2024-02-10T08:11:39.859 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.858+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12} v 0) v1 -- 0x7f80cc067eb0 con 0x7f80cc10c9d0 2024-02-10T08:11:39.859 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.859+0000 7f80c9ffb640 1 -- 172.21.15.70:0/2341917311 <== mon.2 v1:172.21.15.70:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]=0 pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' already exists v812) v1 ==== 183+0+0 (unknown 3112280274 0 0) 0x7f80b40541b0 con 0x7f80cc10c9d0 2024-02-10T08:11:39.860 INFO:tasks.workunit.client.0.smithi070.stderr:pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' already exists 2024-02-10T08:11:39.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.861+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 >> v1:172.21.15.70:6800/4101821536 conn(0x7f80b806e730 legacy=0x7f80b8070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:39.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.861+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 >> v1:172.21.15.70:6790/0 conn(0x7f80cc10c9d0 legacy=0x7f80cc1a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:39.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.861+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 shutdown_connections 2024-02-10T08:11:39.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.861+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 >> 172.21.15.70:0/2341917311 conn(0x7f80cc076c40 msgr2=0x7f80cc10f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:11:39.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.861+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 shutdown_connections 2024-02-10T08:11:39.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.861+0000 7f80d300e640 1 -- 172.21.15.70:0/2341917311 wait complete. 2024-02-10T08:11:39.871 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 939b67e9-3d3f-4cc4-aedd-d4506338adca max_objects 10 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.953+0000 7f7ad784d640 1 -- 172.21.15.70:0/2878013321 >> v1:172.21.15.70:6790/0 conn(0x7f7ad010c9f0 legacy=0x7f7ad010eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.953+0000 7f7ad784d640 1 -- 172.21.15.70:0/2878013321 shutdown_connections 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.953+0000 7f7ad784d640 1 -- 172.21.15.70:0/2878013321 >> 172.21.15.70:0/2878013321 conn(0x7f7ad00752b0 msgr2=0x7f7ad00756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.953+0000 7f7ad784d640 1 -- 172.21.15.70:0/2878013321 shutdown_connections 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.953+0000 7f7ad784d640 1 -- 172.21.15.70:0/2878013321 wait complete. 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad784d640 1 Processor -- start 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad784d640 1 -- start start 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad784d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ad01a5f30 con 0x7f7ad0100ba0 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad784d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ad01a7130 con 0x7f7ad010c9f0 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad784d640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ad01a8330 con 0x7f7ad0108e80 2024-02-10T08:11:39.957 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad55e9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7ad0100ba0 0x7f7ad0100130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39732/0 (socket says 172.21.15.70:39732) 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad4de8640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f7ad0108e80 0x7f7ad01a0d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:59916/0 (socket says 172.21.15.70:59916) 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ad55e9640 1 -- 172.21.15.70:0/4205729049 learned_addr learned my addr 172.21.15.70:0/4205729049 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133478666 0 0) 0x7f7ad01a5f30 con 0x7f7ad0100ba0 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7abc003520 con 0x7f7ad0100ba0 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899008384 0 0) 0x7f7ad01a8330 con 0x7f7ad0108e80 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.954+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad01a5f30 con 0x7f7ad0108e80 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963931509 0 0) 0x7f7ad01a7130 con 0x7f7ad010c9f0 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad01a8330 con 0x7f7ad010c9f0 2024-02-10T08:11:39.958 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 495029679 0 0) 0x7f7abc003520 con 0x7f7ad0100ba0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ad01a7130 con 0x7f7ad0100ba0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7acc0018b0 con 0x7f7ad0100ba0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4129255661 0 0) 0x7f7ad01a5f30 con 0x7f7ad0108e80 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7abc003520 con 0x7f7ad0108e80 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7ac00018a0 con 0x7f7ad0108e80 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1279757910 0 0) 0x7f7ad01a8330 con 0x7f7ad010c9f0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ad01a5f30 con 0x7f7ad010c9f0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1141535450 0 0) 0x7f7ad01a7130 con 0x7f7ad0100ba0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 >> v1:172.21.15.70:6790/0 conn(0x7f7ad0108e80 legacy=0x7f7ad01a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 >> v1:172.21.15.155:6789/0 conn(0x7f7ad010c9f0 legacy=0x7f7ad01a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad01a9530 con 0x7f7ad0100ba0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad01a8520 con 0x7f7ad0100ba0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad01a89b0 con 0x7f7ad0100ba0 2024-02-10T08:11:39.959 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f7acc001c60 con 0x7f7ad0100ba0 2024-02-10T08:11:39.960 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.955+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f7acc005c00 con 0x7f7ad0100ba0 2024-02-10T08:11:39.960 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.956+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab4005080 con 0x7f7ad0100ba0 2024-02-10T08:11:39.960 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.956+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f7acc003940 con 0x7f7ad0100ba0 2024-02-10T08:11:39.960 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.957+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(812..812 src has 252..812) v4 ==== 7805+0+0 (unknown 3937185038 0 0) 0x7f7acc088ea0 con 0x7f7ad0100ba0 2024-02-10T08:11:39.962 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:39.962+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f7acc055ba0 con 0x7f7ad0100ba0 2024-02-10T08:11:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:39 smithi155 ceph-mon[32688]: pgmap v1730: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:40.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:39 smithi155 ceph-mon[32688]: osdmap e811: 8 total, 8 up, 8 in 2024-02-10T08:11:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:39 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2341917311' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:40.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:39 smithi155 ceph-mon[32688]: from='client.58586 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:40.139 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:40.139+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f7ab4005330 con 0x7f7ad0100ba0 2024-02-10T08:11:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[29548]: pgmap v1730: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[29548]: osdmap e811: 8 total, 8 up, 8 in 2024-02-10T08:11:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2341917311' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:40.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[29548]: from='client.58586 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[39923]: pgmap v1730: 196 pgs: 196 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:11:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[39923]: osdmap e811: 8 total, 8 up, 8 in 2024-02-10T08:11:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2341917311' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:39 smithi070 ceph-mon[39923]: from='client.58586 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:40.755 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:40.755+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v813) v1 ==== 223+0+0 (unknown 2726126357 0 0) 0x7f7acc053f10 con 0x7f7ad0100ba0 2024-02-10T08:11:40.756 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:11:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:11:40.870 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:40.869+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f7ab4001090 con 0x7f7ad0100ba0 2024-02-10T08:11:41.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[29548]: from='client.58586 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]': finished 2024-02-10T08:11:41.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[29548]: osdmap e812: 8 total, 8 up, 8 in 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2341917311' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[29548]: from='client.58586 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[39923]: from='client.58586 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]': finished 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[39923]: osdmap e812: 8 total, 8 up, 8 in 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2341917311' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[39923]: from='client.58586 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:41.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:40 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:11:41.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:40 smithi155 ceph-mon[32688]: from='client.58586 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]': finished 2024-02-10T08:11:41.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:40 smithi155 ceph-mon[32688]: osdmap e812: 8 total, 8 up, 8 in 2024-02-10T08:11:41.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2341917311' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:41.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:40 smithi155 ceph-mon[32688]: from='client.58586 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pg_num": 12}]: dispatch 2024-02-10T08:11:41.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:40 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:11:41.763 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.762+0000 7f7ac67fc640 1 -- 172.21.15.70:0/4205729049 <== mon.0 v1:172.21.15.70:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v814) v1 ==== 223+0+0 (unknown 2041810053 0 0) 0x7f7acc055730 con 0x7f7ad0100ba0 2024-02-10T08:11:41.763 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_objects = 10 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:11:41.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.765+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 >> v1:172.21.15.70:6800/4101821536 conn(0x7f7abc06e930 legacy=0x7f7abc070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:41.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.765+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 >> v1:172.21.15.70:6789/0 conn(0x7f7ad0100ba0 legacy=0x7f7ad0100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:41.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.765+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 shutdown_connections 2024-02-10T08:11:41.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.765+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 >> 172.21.15.70:0/4205729049 conn(0x7f7ad00752b0 msgr2=0x7f7ad01100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:11:41.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.765+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 shutdown_connections 2024-02-10T08:11:41.767 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.765+0000 7f7ad784d640 1 -- 172.21.15.70:0/4205729049 wait complete. 2024-02-10T08:11:41.776 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool application enable 939b67e9-3d3f-4cc4-aedd-d4506338adca rados 2024-02-10T08:11:41.860 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.857+0000 7f836c1bc640 1 -- 172.21.15.70:0/3430872391 >> v1:172.21.15.70:6789/0 conn(0x7f83641001d0 legacy=0x7f83641005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.857+0000 7f836c1bc640 1 -- 172.21.15.70:0/3430872391 shutdown_connections 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.857+0000 7f836c1bc640 1 -- 172.21.15.70:0/3430872391 >> 172.21.15.70:0/3430872391 conn(0x7f83640fba30 msgr2=0x7f83640fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.857+0000 7f836c1bc640 1 -- 172.21.15.70:0/3430872391 shutdown_connections 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.857+0000 7f836c1bc640 1 -- 172.21.15.70:0/3430872391 wait complete. 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f836c1bc640 1 Processor -- start 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f836c1bc640 1 -- start start 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f836c1bc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f83641a5f60 con 0x7f83641001d0 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f836c1bc640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f83641a7120 con 0x7f836410c9c0 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f836c1bc640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f83641a8320 con 0x7f8364108e50 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f8369f58640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f83641001d0 0x7f8364069fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39752/0 (socket says 172.21.15.70:39752) 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f8369757640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f8364108e50 0x7f83641a0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:59938/0 (socket says 172.21.15.70:59938) 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f8369f58640 1 -- 172.21.15.70:0/214620536 learned_addr learned my addr 172.21.15.70:0/214620536 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:11:41.861 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305291236 0 0) 0x7f83641a5f60 con 0x7f83641001d0 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8350003520 con 0x7f83641001d0 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177682789 0 0) 0x7f83641a8320 con 0x7f8364108e50 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.858+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83641a5f60 con 0x7f8364108e50 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 879793558 0 0) 0x7f83641a5f60 con 0x7f8364108e50 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83641a8320 con 0x7f8364108e50 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f83540018b0 con 0x7f8364108e50 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1556086012 0 0) 0x7f8350003520 con 0x7f83641001d0 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83641a5f60 con 0x7f83641001d0 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f83600043f0 con 0x7f83641001d0 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135408353 0 0) 0x7f83641a7120 con 0x7f836410c9c0 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8350003520 con 0x7f836410c9c0 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3460555317 0 0) 0x7f83641a8320 con 0x7f8364108e50 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 >> v1:172.21.15.155:6789/0 conn(0x7f836410c9c0 legacy=0x7f83641a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:41.862 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 >> v1:172.21.15.70:6789/0 conn(0x7f83641001d0 legacy=0x7f8364069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:41.863 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83641a9520 con 0x7f8364108e50 2024-02-10T08:11:41.863 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83641a8510 con 0x7f8364108e50 2024-02-10T08:11:41.863 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f83641a89a0 con 0x7f8364108e50 2024-02-10T08:11:41.863 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f8354003aa0 con 0x7f8364108e50 2024-02-10T08:11:41.863 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.859+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8354005ba0 con 0x7f8364108e50 2024-02-10T08:11:41.863 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.860+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8348005080 con 0x7f8364108e50 2024-02-10T08:11:41.863 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.860+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f83540035f0 con 0x7f8364108e50 2024-02-10T08:11:41.864 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.861+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(814..814 src has 252..814) v4 ==== 7805+0+0 (unknown 2281247080 0 0) 0x7f8354088f20 con 0x7f8364108e50 2024-02-10T08:11:41.866 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:41.865+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8354055c20 con 0x7f8364108e50 2024-02-10T08:11:42.042 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:42.042+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"} v 0) v1 -- 0x7f8348005330 con 0x7f8364108e50 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[39923]: pgmap v1733: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[39923]: osdmap e813: 8 total, 8 up, 8 in 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[29548]: pgmap v1733: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:11:42.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:11:42.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[29548]: osdmap e813: 8 total, 8 up, 8 in 2024-02-10T08:11:42.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:11:42.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:41 smithi155 ceph-mon[32688]: pgmap v1733: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:11:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:11:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:41 smithi155 ceph-mon[32688]: osdmap e813: 8 total, 8 up, 8 in 2024-02-10T08:11:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:41 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:11:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:11:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:11:42.771 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:42.770+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]=0 enabled application 'rados' on pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' v815) v1 ==== 213+0+0 (unknown 1991551045 0 0) 0x7f8354053f90 con 0x7f8364108e50 2024-02-10T08:11:42.883 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:42.882+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"} v 0) v1 -- 0x7f8348001090 con 0x7f8364108e50 2024-02-10T08:11:43.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:11:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[39923]: osdmap e814: 8 total, 8 up, 8 in 2024-02-10T08:11:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/214620536' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:43.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[39923]: from='client.58598 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:11:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[29548]: osdmap e814: 8 total, 8 up, 8 in 2024-02-10T08:11:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/214620536' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:43.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:42 smithi070 ceph-mon[29548]: from='client.58598 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:43.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4205729049' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:11:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:42 smithi155 ceph-mon[32688]: osdmap e814: 8 total, 8 up, 8 in 2024-02-10T08:11:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:42 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/214620536' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:43.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:42 smithi155 ceph-mon[32688]: from='client.58598 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:43.772 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:43.771+0000 7f835affd640 1 -- 172.21.15.70:0/214620536 <== mon.2 v1:172.21.15.70:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]=0 enabled application 'rados' on pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' v816) v1 ==== 213+0+0 (unknown 2828622917 0 0) 0x7f83540557b0 con 0x7f8364108e50 2024-02-10T08:11:43.773 INFO:tasks.workunit.client.0.smithi070.stderr:enabled application 'rados' on pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' 2024-02-10T08:11:43.775 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:43.774+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 >> v1:172.21.15.70:6800/4101821536 conn(0x7f835006e880 legacy=0x7f8350070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:43.776 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:43.775+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 >> v1:172.21.15.70:6790/0 conn(0x7f8364108e50 legacy=0x7f83641a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:11:43.776 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:43.775+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 shutdown_connections 2024-02-10T08:11:43.776 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:43.775+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 >> 172.21.15.70:0/214620536 conn(0x7f83640fba30 msgr2=0x7f836410fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:11:43.776 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:43.775+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 shutdown_connections 2024-02-10T08:11:43.776 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:11:43.775+0000 7f836c1bc640 1 -- 172.21.15.70:0/214620536 wait complete. 2024-02-10T08:11:43.787 INFO:tasks.workunit.client.0.smithi070.stderr:++ seq 1 10 2024-02-10T08:11:43.799 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:43.799 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj1 /etc/passwd 2024-02-10T08:11:43.837 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:43.837 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj2 /etc/passwd 2024-02-10T08:11:43.872 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:43.872 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj3 /etc/passwd 2024-02-10T08:11:43.904 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:43.904 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj4 /etc/passwd 2024-02-10T08:11:43.937 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:43.937 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj5 /etc/passwd 2024-02-10T08:11:43.972 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:43.972 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj6 /etc/passwd 2024-02-10T08:11:44.004 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:44.004 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj7 /etc/passwd 2024-02-10T08:11:44.036 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:44.036 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj8 /etc/passwd 2024-02-10T08:11:44.068 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:44.068 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj9 /etc/passwd 2024-02-10T08:11:44.100 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:11:44.100 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put obj10 /etc/passwd 2024-02-10T08:11:44.132 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 30 2024-02-10T08:11:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[29548]: pgmap v1736: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[29548]: from='client.58598 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]': finished 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[29548]: osdmap e815: 8 total, 8 up, 8 in 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/214620536' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[29548]: from='client.58598 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[39923]: pgmap v1736: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[39923]: from='client.58598 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]': finished 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[39923]: osdmap e815: 8 total, 8 up, 8 in 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/214620536' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:43 smithi070 ceph-mon[39923]: from='client.58598 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:43 smithi155 ceph-mon[32688]: pgmap v1736: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 902 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:43 smithi155 ceph-mon[32688]: from='client.58598 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]': finished 2024-02-10T08:11:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:43 smithi155 ceph-mon[32688]: osdmap e815: 8 total, 8 up, 8 in 2024-02-10T08:11:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:43 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/214620536' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:43 smithi155 ceph-mon[32688]: from='client.58598 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]: dispatch 2024-02-10T08:11:45.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:44 smithi070 ceph-mon[29548]: from='client.58598 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]': finished 2024-02-10T08:11:45.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:44 smithi070 ceph-mon[29548]: osdmap e816: 8 total, 8 up, 8 in 2024-02-10T08:11:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:44 smithi070 ceph-mon[39923]: from='client.58598 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]': finished 2024-02-10T08:11:45.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:44 smithi070 ceph-mon[39923]: osdmap e816: 8 total, 8 up, 8 in 2024-02-10T08:11:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:44 smithi155 ceph-mon[32688]: from='client.58598 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "app": "rados"}]': finished 2024-02-10T08:11:45.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:44 smithi155 ceph-mon[32688]: osdmap e816: 8 total, 8 up, 8 in 2024-02-10T08:11:46.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:45 smithi070 ceph-mon[29548]: pgmap v1739: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:46.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:45 smithi070 ceph-mon[29548]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:11:46.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:45 smithi070 ceph-mon[39923]: pgmap v1739: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:46.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:45 smithi070 ceph-mon[39923]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:11:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:45 smithi155 ceph-mon[32688]: pgmap v1739: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:11:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:45 smithi155 ceph-mon[32688]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:11:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:47 smithi070 ceph-mon[29548]: pgmap v1740: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-02-10T08:11:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:47 smithi070 ceph-mon[39923]: pgmap v1740: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-02-10T08:11:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:47 smithi155 ceph-mon[32688]: pgmap v1740: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-02-10T08:11:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:49 smithi070 ceph-mon[29548]: pgmap v1741: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 759 B/s rd, 0 op/s 2024-02-10T08:11:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:49 smithi070 ceph-mon[39923]: pgmap v1741: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 759 B/s rd, 0 op/s 2024-02-10T08:11:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:49 smithi155 ceph-mon[32688]: pgmap v1741: 176 pgs: 176 active+clean; 582 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 759 B/s rd, 0 op/s 2024-02-10T08:11:50.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:11:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:11:52.044 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:52.044 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:51 smithi070 ceph-mon[39923]: pgmap v1742: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:11:52.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:52.044 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:51 smithi070 ceph-mon[29548]: pgmap v1742: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:11:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:11:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:51 smithi155 ceph-mon[32688]: pgmap v1742: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:11:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:11:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:11:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:11:53.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:52 smithi070 ceph-mon[29548]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' is full (reached quota's max_objects: 10) 2024-02-10T08:11:53.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:52 smithi070 ceph-mon[29548]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:11:53.056 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:52 smithi070 ceph-mon[29548]: osdmap e817: 8 total, 8 up, 8 in 2024-02-10T08:11:53.056 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:52 smithi070 ceph-mon[39923]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' is full (reached quota's max_objects: 10) 2024-02-10T08:11:53.056 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:52 smithi070 ceph-mon[39923]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:11:53.056 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:52 smithi070 ceph-mon[39923]: osdmap e817: 8 total, 8 up, 8 in 2024-02-10T08:11:53.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:52 smithi155 ceph-mon[32688]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' is full (reached quota's max_objects: 10) 2024-02-10T08:11:53.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:52 smithi155 ceph-mon[32688]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:11:53.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:52 smithi155 ceph-mon[32688]: osdmap e817: 8 total, 8 up, 8 in 2024-02-10T08:11:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:53 smithi070 ceph-mon[39923]: pgmap v1743: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-02-10T08:11:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:53 smithi070 ceph-mon[29548]: pgmap v1743: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-02-10T08:11:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:53 smithi155 ceph-mon[32688]: pgmap v1743: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-02-10T08:11:56.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:55 smithi070 ceph-mon[39923]: pgmap v1745: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:11:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:55 smithi070 ceph-mon[29548]: pgmap v1745: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:11:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:55 smithi155 ceph-mon[32688]: pgmap v1745: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:11:57.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:57 smithi070 ceph-mon[29548]: pgmap v1746: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:11:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:11:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:57 smithi070 ceph-mon[39923]: pgmap v1746: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:11:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:11:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:11:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:57 smithi155 ceph-mon[32688]: pgmap v1746: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:11:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:11:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:00.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:11:59 smithi155 ceph-mon[32688]: pgmap v1747: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:12:00.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:11:59 smithi070 ceph-mon[39923]: pgmap v1747: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:12:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:11:59 smithi070 ceph-mon[29548]: pgmap v1747: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:12:00.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:12:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:12:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:01 smithi070 ceph-mon[29548]: pgmap v1748: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:01.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:01 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:12:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:01 smithi070 ceph-mon[39923]: pgmap v1748: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:01.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:01 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:12:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:01 smithi155 ceph-mon[32688]: pgmap v1748: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:02.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:01 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:12:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:12:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:12:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:12:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:03 smithi155 ceph-mon[32688]: pgmap v1749: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:04.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:03 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:04.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:03 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:04.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:03 smithi070 ceph-mon[29548]: pgmap v1749: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:04.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:03 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:04.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:03 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:03 smithi070 ceph-mon[39923]: pgmap v1749: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:03 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:03 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:05 smithi155 ceph-mon[32688]: pgmap v1750: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:12:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:05 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:05 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:05 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:12:06.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:05 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:12:06.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:05 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[29548]: pgmap v1750: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:12:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[39923]: pgmap v1750: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:12:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:05 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:08.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:07 smithi155 ceph-mon[32688]: pgmap v1751: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:08.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:07 smithi070 ceph-mon[29548]: pgmap v1751: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:07 smithi070 ceph-mon[39923]: pgmap v1751: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:10.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:09 smithi155 ceph-mon[32688]: pgmap v1752: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:10.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:09 smithi070 ceph-mon[29548]: pgmap v1752: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:09 smithi070 ceph-mon[39923]: pgmap v1752: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:10.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:12:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:12:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:11 smithi070 ceph-mon[39923]: pgmap v1753: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:12.063 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:12.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:11 smithi070 ceph-mon[29548]: pgmap v1753: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:12.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:12.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:11 smithi155 ceph-mon[32688]: pgmap v1753: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:12.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:12:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:12:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:12:14.134 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=154534 2024-02-10T08:12:14.134 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 939b67e9-3d3f-4cc4-aedd-d4506338adca max_objects 100 2024-02-10T08:12:14.134 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put onemore /etc/passwd 2024-02-10T08:12:14.231 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.227+0000 7f8e82e2e640 1 -- 172.21.15.70:0/748852424 >> v1:172.21.15.70:6790/0 conn(0x7f8e7c102250 legacy=0x7f8e7c102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:14.231 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- 172.21.15.70:0/748852424 shutdown_connections 2024-02-10T08:12:14.231 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- 172.21.15.70:0/748852424 >> 172.21.15.70:0/748852424 conn(0x7f8e7c0fba90 msgr2=0x7f8e7c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:14.231 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- 172.21.15.70:0/748852424 shutdown_connections 2024-02-10T08:12:14.231 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- 172.21.15.70:0/748852424 wait complete. 2024-02-10T08:12:14.231 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 Processor -- start 2024-02-10T08:12:14.231 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- start start 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8e7c1a5fa0 con 0x7f8e7c10db60 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8e7c1a71a0 con 0x7f8e7c102250 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e82e2e640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8e7c1a83a0 con 0x7f8e7c10a050 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e73fff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f8e7c10a050 0x7f8e7c1a0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:53338/0 (socket says 172.21.15.70:53338) 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.228+0000 7f8e73fff640 1 -- 172.21.15.70:0/2818795062 learned_addr learned my addr 172.21.15.70:0/2818795062 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039838340 0 0) 0x7f8e7c1a5fa0 con 0x7f8e7c10db60 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e64004950 con 0x7f8e7c10db60 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783755243 0 0) 0x7f8e7c1a83a0 con 0x7f8e7c10a050 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e7c1a5fa0 con 0x7f8e7c10a050 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742136941 0 0) 0x7f8e7c1a71a0 con 0x7f8e7c102250 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e7c1a83a0 con 0x7f8e7c102250 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1002781767 0 0) 0x7f8e64004950 con 0x7f8e7c10db60 2024-02-10T08:12:14.232 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e7c1a71a0 con 0x7f8e7c10db60 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3912193600 0 0) 0x7f8e7c1a5fa0 con 0x7f8e7c10a050 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e64004950 con 0x7f8e7c10a050 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8e74001d70 con 0x7f8e7c10db60 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8e6c001870 con 0x7f8e7c10a050 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2327062651 0 0) 0x7f8e7c1a83a0 con 0x7f8e7c102250 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e7c1a5fa0 con 0x7f8e7c102250 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8e780019c0 con 0x7f8e7c102250 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3943970641 0 0) 0x7f8e7c1a71a0 con 0x7f8e7c10db60 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 >> v1:172.21.15.70:6790/0 conn(0x7f8e7c10a050 legacy=0x7f8e7c1a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 >> v1:172.21.15.155:6789/0 conn(0x7f8e7c102250 legacy=0x7f8e7c109830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c1a95a0 con 0x7f8e7c10db60 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c1a8590 con 0x7f8e7c10db60 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f8e74003e00 con 0x7f8e7c10db60 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8e74004c00 con 0x7f8e7c10db60 2024-02-10T08:12:14.233 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.229+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c1a8a20 con 0x7f8e7c10db60 2024-02-10T08:12:14.234 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.230+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c069f90 con 0x7f8e7c10db60 2024-02-10T08:12:14.234 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.231+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f8e74003970 con 0x7f8e7c10db60 2024-02-10T08:12:14.234 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.231+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(817..817 src has 252..817) v4 ==== 7818+0+0 (unknown 1216753709 0 0) 0x7f8e74087fd0 con 0x7f8e7c10db60 2024-02-10T08:12:14.234 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.231+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=818}) v3 -- 0x7f8e7c1a71a0 con 0x7f8e7c10db60 2024-02-10T08:12:14.236 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.236+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8e74054c60 con 0x7f8e7c10db60 2024-02-10T08:12:14.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:13 smithi070 ceph-mon[29548]: pgmap v1754: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:14.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:13 smithi070 ceph-mon[39923]: pgmap v1754: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:13 smithi155 ceph-mon[32688]: pgmap v1754: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:14.412 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.411+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f8e7c107e00 con 0x7f8e7c10db60 2024-02-10T08:12:14.764 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.763+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v818) v1 ==== 225+0+0 (unknown 4060946439 0 0) 0x7f8e74052fd0 con 0x7f8e7c10db60 2024-02-10T08:12:14.764 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.763+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 11 ==== osd_map(818..818 src has 252..818) v4 ==== 628+0+0 (unknown 4111851109 0 0) 0x7f8e74055590 con 0x7f8e7c10db60 2024-02-10T08:12:14.764 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.763+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=819}) v3 -- 0x7f8e64004950 con 0x7f8e7c10db60 2024-02-10T08:12:14.873 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:14.873+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f8e7c068300 con 0x7f8e7c10db60 2024-02-10T08:12:15.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:14 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:12:15.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:14 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:12:15.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:14 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:12:15.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.765+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v819) v1 ==== 225+0+0 (unknown 3673037951 0 0) 0x7f8e740547f0 con 0x7f8e7c10db60 2024-02-10T08:12:15.766 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_objects = 100 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:12:15.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.765+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 <== mon.0 v1:172.21.15.70:6789/0 13 ==== osd_map(819..819 src has 252..819) v4 ==== 628+0+0 (unknown 1898898832 0 0) 0x7f8e74058fe0 con 0x7f8e7c10db60 2024-02-10T08:12:15.766 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.765+0000 7f8e71ffb640 1 -- 172.21.15.70:0/2818795062 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=820}) v3 -- 0x7f8e7c1a83a0 con 0x7f8e7c10db60 2024-02-10T08:12:15.770 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.769+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 >> v1:172.21.15.70:6800/4101821536 conn(0x7f8e6406e9c0 legacy=0x7f8e64070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:15.770 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.769+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 >> v1:172.21.15.70:6789/0 conn(0x7f8e7c10db60 legacy=0x7f8e7c1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:15.770 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.770+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 shutdown_connections 2024-02-10T08:12:15.771 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.770+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 >> 172.21.15.70:0/2818795062 conn(0x7f8e7c0fba90 msgr2=0x7f8e7c10ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:15.771 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.770+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 shutdown_connections 2024-02-10T08:12:15.771 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:15.770+0000 7f8e82e2e640 1 -- 172.21.15.70:0/2818795062 wait complete. 2024-02-10T08:12:15.785 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 154534 2024-02-10T08:12:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[29548]: pgmap v1755: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:16.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:12:16.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[29548]: osdmap e818: 8 total, 8 up, 8 in 2024-02-10T08:12:16.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:12:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[39923]: pgmap v1755: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:12:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[39923]: osdmap e818: 8 total, 8 up, 8 in 2024-02-10T08:12:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:15 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:12:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:15 smithi155 ceph-mon[32688]: pgmap v1755: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:12:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:15 smithi155 ceph-mon[32688]: osdmap e818: 8 total, 8 up, 8 in 2024-02-10T08:12:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:15 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:12:17.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:16 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:12:17.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:16 smithi070 ceph-mon[29548]: osdmap e819: 8 total, 8 up, 8 in 2024-02-10T08:12:17.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:16 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:12:17.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:16 smithi070 ceph-mon[39923]: osdmap e819: 8 total, 8 up, 8 in 2024-02-10T08:12:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:16 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2818795062' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:12:17.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:16 smithi155 ceph-mon[32688]: osdmap e819: 8 total, 8 up, 8 in 2024-02-10T08:12:17.756 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 0 -ne 0 ']' 2024-02-10T08:12:17.756 INFO:tasks.workunit.client.0.smithi070.stderr:+ true 2024-02-10T08:12:17.757 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put twomore /etc/passwd 2024-02-10T08:12:17.796 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 939b67e9-3d3f-4cc4-aedd-d4506338adca max_bytes 100 2024-02-10T08:12:17.883 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.880+0000 7f8cd3c72640 1 -- 172.21.15.70:0/1979193167 >> v1:172.21.15.70:6789/0 conn(0x7f8ccc109f60 legacy=0x7f8ccc10c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.880+0000 7f8cd3c72640 1 -- 172.21.15.70:0/1979193167 shutdown_connections 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.880+0000 7f8cd3c72640 1 -- 172.21.15.70:0/1979193167 >> 172.21.15.70:0/1979193167 conn(0x7f8ccc0fcb60 msgr2=0x7f8ccc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.880+0000 7f8cd3c72640 1 -- 172.21.15.70:0/1979193167 shutdown_connections 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.880+0000 7f8cd3c72640 1 -- 172.21.15.70:0/1979193167 wait complete. 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd3c72640 1 Processor -- start 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd3c72640 1 -- start start 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd3c72640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ccc1a5fa0 con 0x7f8ccc109f60 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd3c72640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ccc1a7160 con 0x7f8ccc1012e0 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd3c72640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ccc1a8360 con 0x7f8ccc10dad0 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd220f640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f8ccc10dad0 0x7f8ccc1a4670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:53444/0 (socket says 172.21.15.70:53444) 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd120d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8ccc109f60 0x7f8ccc1a0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47394/0 (socket says 172.21.15.70:47394) 2024-02-10T08:12:17.884 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cd220f640 1 -- 172.21.15.70:0/2418136524 learned_addr learned my addr 172.21.15.70:0/2418136524 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647597153 0 0) 0x7f8ccc1a8360 con 0x7f8ccc10dad0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb8003160 con 0x7f8ccc10dad0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894522011 0 0) 0x7f8ccc1a5fa0 con 0x7f8ccc109f60 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ccc1a8360 con 0x7f8ccc109f60 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340984370 0 0) 0x7f8ccc1a7160 con 0x7f8ccc1012e0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ccc1a5fa0 con 0x7f8ccc1012e0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 717812921 0 0) 0x7f8cb8003160 con 0x7f8ccc10dad0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ccc1a7160 con 0x7f8ccc10dad0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8cc4001e10 con 0x7f8ccc10dad0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1514519481 0 0) 0x7f8ccc1a8360 con 0x7f8ccc109f60 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8cb8003160 con 0x7f8ccc109f60 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.881+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8cbc0043f0 con 0x7f8ccc109f60 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3895329649 0 0) 0x7f8ccc1a7160 con 0x7f8ccc10dad0 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 >> v1:172.21.15.155:6789/0 conn(0x7f8ccc1012e0 legacy=0x7f8ccc106580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:17.885 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 >> v1:172.21.15.70:6789/0 conn(0x7f8ccc109f60 legacy=0x7f8ccc1a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc1a9560 con 0x7f8ccc10dad0 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc1a6190 con 0x7f8ccc10dad0 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1a6620 con 0x7f8ccc10dad0 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f8cc40020b0 con 0x7f8ccc10dad0 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8cc4005f00 con 0x7f8ccc10dad0 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.882+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb0005080 con 0x7f8ccc10dad0 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.883+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f8cc4003c20 con 0x7f8ccc10dad0 2024-02-10T08:12:17.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.883+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(820..820 src has 252..820) v4 ==== 7818+0+0 (unknown 4000449572 0 0) 0x7f8cc4089240 con 0x7f8ccc10dad0 2024-02-10T08:12:17.888 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:17.888+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f8cc4055ca0 con 0x7f8ccc10dad0 2024-02-10T08:12:18.065 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:18.065+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f8cb0005330 con 0x7f8ccc10dad0 2024-02-10T08:12:18.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[39923]: pgmap v1758: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:18.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[39923]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:12:18.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[39923]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:12:18.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[39923]: osdmap e820: 8 total, 8 up, 8 in 2024-02-10T08:12:18.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[29548]: pgmap v1758: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:18.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[29548]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:12:18.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[29548]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:12:18.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:17 smithi070 ceph-mon[29548]: osdmap e820: 8 total, 8 up, 8 in 2024-02-10T08:12:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:17 smithi155 ceph-mon[32688]: pgmap v1758: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:17 smithi155 ceph-mon[32688]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:12:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:17 smithi155 ceph-mon[32688]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:12:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:17 smithi155 ceph-mon[32688]: osdmap e820: 8 total, 8 up, 8 in 2024-02-10T08:12:18.791 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:18.790+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v821) v1 ==== 221+0+0 (unknown 2545399116 0 0) 0x7f8cc40542a0 con 0x7f8ccc10dad0 2024-02-10T08:12:18.899 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:18.898+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f8cb0000c40 con 0x7f8ccc10dad0 2024-02-10T08:12:19.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:18 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2418136524' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:19.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:18 smithi070 ceph-mon[39923]: from='client.58718 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:19.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:18 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2418136524' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:19.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:18 smithi070 ceph-mon[29548]: from='client.58718 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:19.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:18 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2418136524' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:19.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:18 smithi155 ceph-mon[32688]: from='client.58718 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:19.797 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:19.796+0000 7f8cc2ffd640 1 -- 172.21.15.70:0/2418136524 <== mon.2 v1:172.21.15.70:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v822) v1 ==== 221+0+0 (unknown 313176478 0 0) 0x7f8cc4055ac0 con 0x7f8ccc10dad0 2024-02-10T08:12:19.797 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_bytes = 100 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:12:19.799 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:19.798+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 >> v1:172.21.15.70:6800/4101821536 conn(0x7f8cb806e7a0 legacy=0x7f8cb8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:19.799 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:19.798+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 >> v1:172.21.15.70:6790/0 conn(0x7f8ccc10dad0 legacy=0x7f8ccc1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:19.799 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:19.799+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 shutdown_connections 2024-02-10T08:12:19.800 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:19.799+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 >> 172.21.15.70:0/2418136524 conn(0x7f8ccc0fcb60 msgr2=0x7f8ccc106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:19.800 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:19.799+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 shutdown_connections 2024-02-10T08:12:19.800 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:19.799+0000 7f8cd3c72640 1 -- 172.21.15.70:0/2418136524 wait complete. 2024-02-10T08:12:19.811 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 30 2024-02-10T08:12:20.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[29548]: pgmap v1760: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[29548]: from='client.58718 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[29548]: osdmap e821: 8 total, 8 up, 8 in 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2418136524' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[29548]: from='client.58718 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[39923]: pgmap v1760: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[39923]: from='client.58718 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[39923]: osdmap e821: 8 total, 8 up, 8 in 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2418136524' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:20.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:19 smithi070 ceph-mon[39923]: from='client.58718 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:19 smithi155 ceph-mon[32688]: pgmap v1760: 176 pgs: 176 active+clean; 697 KiB data, 907 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:19 smithi155 ceph-mon[32688]: from='client.58718 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:12:20.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:19 smithi155 ceph-mon[32688]: osdmap e821: 8 total, 8 up, 8 in 2024-02-10T08:12:20.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:19 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2418136524' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:20.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:19 smithi155 ceph-mon[32688]: from='client.58718 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:12:20.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:12:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:12:21.077 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:20 smithi155 ceph-mon[32688]: from='client.58718 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:12:21.077 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:20 smithi155 ceph-mon[32688]: osdmap e822: 8 total, 8 up, 8 in 2024-02-10T08:12:21.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:20 smithi070 ceph-mon[29548]: from='client.58718 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:12:21.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:20 smithi070 ceph-mon[29548]: osdmap e822: 8 total, 8 up, 8 in 2024-02-10T08:12:21.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:20 smithi070 ceph-mon[39923]: from='client.58718 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:12:21.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:20 smithi070 ceph-mon[39923]: osdmap e822: 8 total, 8 up, 8 in 2024-02-10T08:12:22.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:22.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:21 smithi070 ceph-mon[29548]: pgmap v1763: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-02-10T08:12:22.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:22.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:21 smithi070 ceph-mon[39923]: pgmap v1763: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-02-10T08:12:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:21 smithi155 ceph-mon[32688]: pgmap v1763: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-02-10T08:12:22.339 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:12:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:12:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:12:23.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:22 smithi070 ceph-mon[29548]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' is full (reached quota's max_bytes: 100 B) 2024-02-10T08:12:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:22 smithi070 ceph-mon[29548]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:12:23.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:22 smithi070 ceph-mon[29548]: osdmap e823: 8 total, 8 up, 8 in 2024-02-10T08:12:23.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:22 smithi070 ceph-mon[39923]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' is full (reached quota's max_bytes: 100 B) 2024-02-10T08:12:23.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:22 smithi070 ceph-mon[39923]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:12:23.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:22 smithi070 ceph-mon[39923]: osdmap e823: 8 total, 8 up, 8 in 2024-02-10T08:12:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:22 smithi155 ceph-mon[32688]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' is full (reached quota's max_bytes: 100 B) 2024-02-10T08:12:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:22 smithi155 ceph-mon[32688]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:12:23.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:22 smithi155 ceph-mon[32688]: osdmap e823: 8 total, 8 up, 8 in 2024-02-10T08:12:24.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:23 smithi070 ceph-mon[29548]: pgmap v1764: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T08:12:24.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:23 smithi070 ceph-mon[39923]: pgmap v1764: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T08:12:24.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:23 smithi155 ceph-mon[32688]: pgmap v1764: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T08:12:26.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:25 smithi070 ceph-mon[39923]: pgmap v1766: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T08:12:26.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:25 smithi070 ceph-mon[29548]: pgmap v1766: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T08:12:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:25 smithi155 ceph-mon[32688]: pgmap v1766: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-10T08:12:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:28.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:27 smithi070 ceph-mon[29548]: pgmap v1767: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-02-10T08:12:28.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:27 smithi070 ceph-mon[39923]: pgmap v1767: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-02-10T08:12:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:27 smithi155 ceph-mon[32688]: pgmap v1767: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 1 op/s 2024-02-10T08:12:29.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:28 smithi070 ceph-mon[29548]: pgmap v1768: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 587 B/s rd, 0 op/s 2024-02-10T08:12:29.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:28 smithi070 ceph-mon[39923]: pgmap v1768: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 587 B/s rd, 0 op/s 2024-02-10T08:12:29.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:28 smithi155 ceph-mon[32688]: pgmap v1768: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 587 B/s rd, 0 op/s 2024-02-10T08:12:30.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:12:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:12:31.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:31.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:31 smithi070 ceph-mon[39923]: pgmap v1769: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:31.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:31 smithi070 ceph-mon[29548]: pgmap v1769: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:32.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:32.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:31 smithi155 ceph-mon[32688]: pgmap v1769: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:12:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:12:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:12:34.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:33 smithi155 ceph-mon[32688]: pgmap v1770: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:33 smithi070 ceph-mon[29548]: pgmap v1770: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:33 smithi070 ceph-mon[39923]: pgmap v1770: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:12:36.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:35 smithi155 ceph-mon[32688]: pgmap v1771: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:12:36.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:35 smithi070 ceph-mon[29548]: pgmap v1771: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:12:36.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:35 smithi070 ceph-mon[39923]: pgmap v1771: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:12:38.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:37 smithi155 ceph-mon[32688]: pgmap v1772: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:38.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:37 smithi070 ceph-mon[29548]: pgmap v1772: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:37 smithi070 ceph-mon[39923]: pgmap v1772: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:39 smithi070 ceph-mon[29548]: pgmap v1773: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:39 smithi070 ceph-mon[39923]: pgmap v1773: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:40.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:39 smithi155 ceph-mon[32688]: pgmap v1773: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:40.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:12:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:12:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:41 smithi070 ceph-mon[39923]: pgmap v1774: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:42.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:42.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:42.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:41 smithi070 ceph-mon[29548]: pgmap v1774: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:42.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:41 smithi155 ceph-mon[32688]: pgmap v1774: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:42.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:12:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:12:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:12:44.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:43 smithi070 ceph-mon[29548]: pgmap v1775: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:44.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:43 smithi070 ceph-mon[39923]: pgmap v1775: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:44.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:43 smithi155 ceph-mon[32688]: pgmap v1775: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:46.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:45 smithi070 ceph-mon[39923]: pgmap v1776: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:46.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:45 smithi070 ceph-mon[29548]: pgmap v1776: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:45 smithi155 ceph-mon[32688]: pgmap v1776: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:47 smithi070 ceph-mon[29548]: pgmap v1777: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:47 smithi070 ceph-mon[39923]: pgmap v1777: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:47 smithi155 ceph-mon[32688]: pgmap v1777: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:49.813 INFO:tasks.workunit.client.0.smithi070.stderr:+ pid=154634 2024-02-10T08:12:49.813 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 939b67e9-3d3f-4cc4-aedd-d4506338adca max_bytes 0 2024-02-10T08:12:49.813 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put two /etc/passwd 2024-02-10T08:12:49.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.903+0000 7fcafa488640 1 -- 172.21.15.70:0/675075625 >> v1:172.21.15.70:6789/0 conn(0x7fcaf410a030 legacy=0x7fcaf410c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:49.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.903+0000 7fcafa488640 1 -- 172.21.15.70:0/675075625 shutdown_connections 2024-02-10T08:12:49.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.903+0000 7fcafa488640 1 -- 172.21.15.70:0/675075625 >> 172.21.15.70:0/675075625 conn(0x7fcaf40fba30 msgr2=0x7fcaf40fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:49.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.903+0000 7fcafa488640 1 -- 172.21.15.70:0/675075625 shutdown_connections 2024-02-10T08:12:49.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.904+0000 7fcafa488640 1 -- 172.21.15.70:0/675075625 wait complete. 2024-02-10T08:12:49.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.904+0000 7fcafa488640 1 Processor -- start 2024-02-10T08:12:49.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.904+0000 7fcafa488640 1 -- start start 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.904+0000 7fcafa488640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcaf41a5f20 con 0x7fcaf4102230 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.904+0000 7fcafa488640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcaf41a7120 con 0x7fcaf410db40 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.904+0000 7fcafa488640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcaf41a8320 con 0x7fcaf410a030 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcaf4102230 0x7fcaf41097b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39560/0 (socket says 172.21.15.70:39560) 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf3fff640 1 -- 172.21.15.70:0/1750724314 learned_addr learned my addr 172.21.15.70:0/1750724314 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf37fe640 1 --1- 172.21.15.70:0/1750724314 >> v1:172.21.15.70:6790/0 conn(0x7fcaf410a030 0x7fcaf41a0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48256/0 (socket says 172.21.15.70:48256) 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890863828 0 0) 0x7fcaf41a8320 con 0x7fcaf410a030 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc003520 con 0x7fcaf410a030 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495078795 0 0) 0x7fcaf41a5f20 con 0x7fcaf4102230 2024-02-10T08:12:49.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf41a8320 con 0x7fcaf4102230 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661160729 0 0) 0x7fcaf41a7120 con 0x7fcaf410db40 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf41a5f20 con 0x7fcaf410db40 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2948370378 0 0) 0x7fcadc003520 con 0x7fcaf410a030 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcaf41a7120 con 0x7fcaf410a030 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcae8004350 con 0x7fcaf410a030 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3046592936 0 0) 0x7fcaf41a8320 con 0x7fcaf4102230 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcadc003520 con 0x7fcaf4102230 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcae40018d0 con 0x7fcaf4102230 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3828078474 0 0) 0x7fcaf41a7120 con 0x7fcaf410a030 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 >> v1:172.21.15.155:6789/0 conn(0x7fcaf410db40 legacy=0x7fcaf41a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 >> v1:172.21.15.70:6789/0 conn(0x7fcaf4102230 legacy=0x7fcaf41097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf41a9520 con 0x7fcaf410a030 2024-02-10T08:12:49.909 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf41a6110 con 0x7fcaf410a030 2024-02-10T08:12:49.910 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf41a65a0 con 0x7fcaf410a030 2024-02-10T08:12:49.910 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fcae8001ab0 con 0x7fcaf410a030 2024-02-10T08:12:49.910 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.905+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fcae8004dc0 con 0x7fcaf410a030 2024-02-10T08:12:49.910 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.906+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae0005080 con 0x7fcaf410a030 2024-02-10T08:12:49.910 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.907+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fcae8005000 con 0x7fcaf410a030 2024-02-10T08:12:49.910 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.907+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(823..823 src has 252..823) v4 ==== 7818+0+0 (unknown 1887815052 0 0) 0x7fcae8088fd0 con 0x7fcaf410a030 2024-02-10T08:12:49.910 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.907+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=824}) v3 -- 0x7fcaf41a7120 con 0x7fcaf410a030 2024-02-10T08:12:49.912 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:49.912+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fcae8055da0 con 0x7fcaf410a030 2024-02-10T08:12:50.088 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:50.087+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fcae0005330 con 0x7fcaf410a030 2024-02-10T08:12:50.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:49 smithi070 ceph-mon[29548]: pgmap v1778: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:49 smithi070 ceph-mon[39923]: pgmap v1778: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:49 smithi155 ceph-mon[32688]: pgmap v1778: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:12:50.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:12:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:12:50.780 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:50.779+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 10 ==== osd_map(824..824 src has 252..824) v4 ==== 628+0+0 (unknown 4108387618 0 0) 0x7fcae80566d0 con 0x7fcaf410a030 2024-02-10T08:12:50.780 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:50.779+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=825}) v3 -- 0x7fcadc003520 con 0x7fcaf410a030 2024-02-10T08:12:50.784 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:50.783+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v824) v1 ==== 217+0+0 (unknown 735886362 0 0) 0x7fcae8054110 con 0x7fcaf410a030 2024-02-10T08:12:50.892 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:50.891+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fcae0000c40 con 0x7fcaf410a030 2024-02-10T08:12:51.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:50 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1750724314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:51.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:50 smithi070 ceph-mon[39923]: from='client.58733 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:50 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1750724314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:51.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:50 smithi070 ceph-mon[29548]: from='client.58733 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:50 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1750724314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:51.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:50 smithi155 ceph-mon[32688]: from='client.58733 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:51.781 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.780+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 12 ==== osd_map(825..825 src has 252..825) v4 ==== 628+0+0 (unknown 1188885031 0 0) 0x7fcae805a120 con 0x7fcaf410a030 2024-02-10T08:12:51.781 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.780+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=826}) v3 -- 0x7fcaf41a5f20 con 0x7fcaf410a030 2024-02-10T08:12:51.785 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.784+0000 7fcaf17fa640 1 -- 172.21.15.70:0/1750724314 <== mon.2 v1:172.21.15.70:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v825) v1 ==== 217+0+0 (unknown 4203196884 0 0) 0x7fcae8055930 con 0x7fcaf410a030 2024-02-10T08:12:51.785 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_bytes = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:12:51.791 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.788+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 >> v1:172.21.15.70:6800/4101821536 conn(0x7fcadc06e850 legacy=0x7fcadc070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:51.791 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.788+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 >> v1:172.21.15.70:6790/0 conn(0x7fcaf410a030 legacy=0x7fcaf41a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:51.791 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.789+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 shutdown_connections 2024-02-10T08:12:51.792 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.789+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 >> 172.21.15.70:0/1750724314 conn(0x7fcaf40fba30 msgr2=0x7fcaf40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:51.792 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.789+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 shutdown_connections 2024-02-10T08:12:51.792 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.789+0000 7fcafa488640 1 -- 172.21.15.70:0/1750724314 wait complete. 2024-02-10T08:12:51.801 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 939b67e9-3d3f-4cc4-aedd-d4506338adca max_objects 0 2024-02-10T08:12:51.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- 172.21.15.70:0/39325768 >> v1:172.21.15.70:6790/0 conn(0x7f66340ffaa0 legacy=0x7f66340ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:51.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- 172.21.15.70:0/39325768 shutdown_connections 2024-02-10T08:12:51.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- 172.21.15.70:0/39325768 >> 172.21.15.70:0/39325768 conn(0x7f6634075260 msgr2=0x7f6634075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:51.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- 172.21.15.70:0/39325768 shutdown_connections 2024-02-10T08:12:51.886 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- 172.21.15.70:0/39325768 wait complete. 2024-02-10T08:12:51.887 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 Processor -- start 2024-02-10T08:12:51.887 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- start start 2024-02-10T08:12:51.887 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66341a5fa0 con 0x7f663410dad0 2024-02-10T08:12:51.887 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66341a7160 con 0x7f6634109f90 2024-02-10T08:12:51.887 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.885+0000 7f663afc7640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66341a8360 con 0x7f66340ffaa0 2024-02-10T08:12:51.887 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6638d63640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f66340ffaa0 0x7f6634106560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48282/0 (socket says 172.21.15.70:48282) 2024-02-10T08:12:51.887 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6638d63640 1 -- 172.21.15.70:0/4255382196 learned_addr learned my addr 172.21.15.70:0/4255382196 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:12:51.889 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954113542 0 0) 0x7f66341a8360 con 0x7f66340ffaa0 2024-02-10T08:12:51.889 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6624004950 con 0x7f66340ffaa0 2024-02-10T08:12:51.889 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577369709 0 0) 0x7f66341a5fa0 con 0x7f663410dad0 2024-02-10T08:12:51.889 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66341a8360 con 0x7f663410dad0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 120089907 0 0) 0x7f6624004950 con 0x7f66340ffaa0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f66341a5fa0 con 0x7f66340ffaa0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6628001750 con 0x7f66340ffaa0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132494417 0 0) 0x7f66341a7160 con 0x7f6634109f90 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6624004950 con 0x7f6634109f90 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1060708933 0 0) 0x7f66341a8360 con 0x7f663410dad0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f66341a7160 con 0x7f663410dad0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f662c001bb0 con 0x7f663410dad0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3232470005 0 0) 0x7f66341a5fa0 con 0x7f66340ffaa0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 >> v1:172.21.15.155:6789/0 conn(0x7f6634109f90 legacy=0x7f66341a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 >> v1:172.21.15.70:6789/0 conn(0x7f663410dad0 legacy=0x7f66341a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66341a9560 con 0x7f66340ffaa0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66341a8550 con 0x7f66340ffaa0 2024-02-10T08:12:51.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f6628001970 con 0x7f66340ffaa0 2024-02-10T08:12:51.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.886+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f6628004f60 con 0x7f66340ffaa0 2024-02-10T08:12:51.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.887+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f66341a89e0 con 0x7f66340ffaa0 2024-02-10T08:12:51.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.888+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6634069340 con 0x7f66340ffaa0 2024-02-10T08:12:51.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.888+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f6628005230 con 0x7f66340ffaa0 2024-02-10T08:12:51.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.888+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(825..825 src has 252..825) v4 ==== 7818+0+0 (unknown 3956846590 0 0) 0x7f66280892a0 con 0x7f66340ffaa0 2024-02-10T08:12:51.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.888+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=826}) v3 -- 0x7f66341a5fa0 con 0x7f66340ffaa0 2024-02-10T08:12:51.893 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:51.893+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f6628055f90 con 0x7f66340ffaa0 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[39923]: pgmap v1779: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[39923]: from='client.58733 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[39923]: osdmap e824: 8 total, 8 up, 8 in 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1750724314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[39923]: from='client.58733 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[29548]: pgmap v1779: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[29548]: from='client.58733 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:12:52.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[29548]: osdmap e824: 8 total, 8 up, 8 in 2024-02-10T08:12:52.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1750724314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:52.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:51 smithi070 ceph-mon[29548]: from='client.58733 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:52.072 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:52.072+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f6634104b30 con 0x7f66340ffaa0 2024-02-10T08:12:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:12:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:51 smithi155 ceph-mon[32688]: pgmap v1779: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:12:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:51 smithi155 ceph-mon[32688]: from='client.58733 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:12:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:51 smithi155 ceph-mon[32688]: osdmap e824: 8 total, 8 up, 8 in 2024-02-10T08:12:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:51 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1750724314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:52.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:51 smithi155 ceph-mon[32688]: from='client.58733 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:12:52.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:12:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:12:52.754 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:52.753+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 10 ==== osd_map(826..826 src has 252..826) v4 ==== 628+0+0 (unknown 2814966415 0 0) 0x7f66280568c0 con 0x7f66340ffaa0 2024-02-10T08:12:52.754 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:52.753+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=827}) v3 -- 0x7f66341a8360 con 0x7f66340ffaa0 2024-02-10T08:12:52.759 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:52.758+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v826) v1 ==== 221+0+0 (unknown 1782370266 0 0) 0x7f6628054300 con 0x7f66340ffaa0 2024-02-10T08:12:52.866 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:52.866+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f6634068300 con 0x7f66340ffaa0 2024-02-10T08:12:53.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: from='client.58733 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: osdmap e825: 8 total, 8 up, 8 in 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4255382196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: from='client.58739 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: from='client.58739 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[29548]: osdmap e826: 8 total, 8 up, 8 in 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: from='client.58733 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: osdmap e825: 8 total, 8 up, 8 in 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4255382196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:53.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: from='client.58739 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:53.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:12:53.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:12:53.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: from='client.58739 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:12:53.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:52 smithi070 ceph-mon[39923]: osdmap e826: 8 total, 8 up, 8 in 2024-02-10T08:12:53.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: from='client.58733 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:12:53.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: osdmap e825: 8 total, 8 up, 8 in 2024-02-10T08:12:53.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4255382196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:53.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: from='client.58739 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:53.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:12:53.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:12:53.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: from='client.58739 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:12:53.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:52 smithi155 ceph-mon[32688]: osdmap e826: 8 total, 8 up, 8 in 2024-02-10T08:12:53.795 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.794+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 12 ==== osd_map(827..827 src has 252..827) v4 ==== 628+0+0 (unknown 2868021031 0 0) 0x7f662805a310 con 0x7f66340ffaa0 2024-02-10T08:12:53.799 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.798+0000 7f6631ffb640 1 -- 172.21.15.70:0/4255382196 <== mon.2 v1:172.21.15.70:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v827) v1 ==== 221+0+0 (unknown 3932954171 0 0) 0x7f6628055b20 con 0x7f66340ffaa0 2024-02-10T08:12:53.799 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_objects = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:12:53.803 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.802+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 >> v1:172.21.15.70:6800/4101821536 conn(0x7f662406e900 legacy=0x7f6624070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:53.803 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.802+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 >> v1:172.21.15.70:6790/0 conn(0x7f66340ffaa0 legacy=0x7f6634106560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:53.803 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.802+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 shutdown_connections 2024-02-10T08:12:53.803 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.802+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 >> 172.21.15.70:0/4255382196 conn(0x7f6634075260 msgr2=0x7f663410df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:53.803 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.802+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 shutdown_connections 2024-02-10T08:12:53.804 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.803+0000 7f663afc7640 1 -- 172.21.15.70:0/4255382196 wait complete. 2024-02-10T08:12:53.816 INFO:tasks.workunit.client.0.smithi070.stderr:+ wait 154634 2024-02-10T08:12:53.816 INFO:tasks.workunit.client.0.smithi070.stderr:+ '[' 0 -ne 0 ']' 2024-02-10T08:12:53.816 INFO:tasks.workunit.client.0.smithi070.stderr:+ true 2024-02-10T08:12:53.816 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put three /etc/passwd 2024-02-10T08:12:53.851 INFO:tasks.workunit.client.0.smithi070.stderr:++ uuidgen 2024-02-10T08:12:53.852 INFO:tasks.workunit.client.0.smithi070.stderr:+ pp=8b5e84ed-f339-493c-81c7-098f848f87fd 2024-02-10T08:12:53.852 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool create 8b5e84ed-f339-493c-81c7-098f848f87fd 12 2024-02-10T08:12:53.938 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.934+0000 7ff928a2e640 1 -- 172.21.15.70:0/296543883 >> v1:172.21.15.70:6789/0 conn(0x7ff9241001e0 legacy=0x7ff9241005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:53.979 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.934+0000 7ff928a2e640 1 -- 172.21.15.70:0/296543883 shutdown_connections 2024-02-10T08:12:53.979 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.934+0000 7ff928a2e640 1 -- 172.21.15.70:0/296543883 >> 172.21.15.70:0/296543883 conn(0x7ff924076c40 msgr2=0x7ff9240fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:53.980 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.934+0000 7ff928a2e640 1 -- 172.21.15.70:0/296543883 shutdown_connections 2024-02-10T08:12:53.980 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.934+0000 7ff928a2e640 1 -- 172.21.15.70:0/296543883 wait complete. 2024-02-10T08:12:53.980 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff928a2e640 1 Processor -- start 2024-02-10T08:12:53.980 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff928a2e640 1 -- start start 2024-02-10T08:12:53.980 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff928a2e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9241a5f40 con 0x7ff92410c9d0 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff928a2e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9241a7140 con 0x7ff9241001e0 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff928a2e640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9241a8340 con 0x7ff924108e60 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff91bfff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7ff924108e60 0x7ff9241a0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48330/0 (socket says 172.21.15.70:48330) 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff922d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff92410c9d0 0x7ff9241a4640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39638/0 (socket says 172.21.15.70:39638) 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff91bfff640 1 -- 172.21.15.70:0/3657439572 learned_addr learned my addr 172.21.15.70:0/3657439572 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775206215 0 0) 0x7ff9241a8340 con 0x7ff924108e60 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff910003520 con 0x7ff924108e60 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270423785 0 0) 0x7ff9241a5f40 con 0x7ff92410c9d0 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.935+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9241a8340 con 0x7ff92410c9d0 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276421554 0 0) 0x7ff9241a7140 con 0x7ff9241001e0 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9241a5f40 con 0x7ff9241001e0 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3500572512 0 0) 0x7ff9241a8340 con 0x7ff92410c9d0 2024-02-10T08:12:53.981 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9241a7140 con 0x7ff92410c9d0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff91c001bd0 con 0x7ff92410c9d0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2247982647 0 0) 0x7ff910003520 con 0x7ff924108e60 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9241a8340 con 0x7ff924108e60 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff90c0018b0 con 0x7ff924108e60 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2145195094 0 0) 0x7ff9241a5f40 con 0x7ff9241001e0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff910003520 con 0x7ff9241001e0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff9140043c0 con 0x7ff9241001e0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 375174359 0 0) 0x7ff9241a7140 con 0x7ff92410c9d0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 >> v1:172.21.15.70:6790/0 conn(0x7ff924108e60 legacy=0x7ff9241a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 >> v1:172.21.15.155:6789/0 conn(0x7ff9241001e0 legacy=0x7ff924105480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9241a9540 con 0x7ff92410c9d0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9241a6130 con 0x7ff92410c9d0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9241a65c0 con 0x7ff92410c9d0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7ff91c003e00 con 0x7ff92410c9d0 2024-02-10T08:12:53.982 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.936+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7ff91c005df0 con 0x7ff92410c9d0 2024-02-10T08:12:53.983 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.937+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9040050b0 con 0x7ff92410c9d0 2024-02-10T08:12:53.983 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.937+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7ff91c003970 con 0x7ff92410c9d0 2024-02-10T08:12:53.983 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.938+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(827..827 src has 252..827) v4 ==== 7818+0+0 (unknown 8531662 0 0) 0x7ff91c089050 con 0x7ff92410c9d0 2024-02-10T08:12:53.984 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:53.942+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7ff91c055e20 con 0x7ff92410c9d0 2024-02-10T08:12:54.117 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.116+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12} v 0) v1 -- 0x7ff9040058e0 con 0x7ff92410c9d0 2024-02-10T08:12:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:53 smithi070 ceph-mon[39923]: pgmap v1782: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:12:54.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:53 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/4255382196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:53 smithi070 ceph-mon[39923]: from='client.58739 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:53 smithi070 ceph-mon[29548]: pgmap v1782: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:12:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:53 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/4255382196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:53 smithi070 ceph-mon[29548]: from='client.58739 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:53 smithi155 ceph-mon[32688]: pgmap v1782: 176 pgs: 176 active+clean; 720 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:12:54.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:53 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/4255382196' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:54.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:53 smithi155 ceph-mon[32688]: from='client.58739 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:12:54.808 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.807+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]=0 pool '8b5e84ed-f339-493c-81c7-098f848f87fd' created v828) v1 ==== 176+0+0 (unknown 1567481934 0 0) 0x7ff91c054190 con 0x7ff92410c9d0 2024-02-10T08:12:54.919 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.918+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12} v 0) v1 -- 0x7ff904005aa0 con 0x7ff92410c9d0 2024-02-10T08:12:54.919 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.918+0000 7ff91b7fe640 1 -- 172.21.15.70:0/3657439572 <== mon.0 v1:172.21.15.70:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]=0 pool '8b5e84ed-f339-493c-81c7-098f848f87fd' already exists v828) v1 ==== 183+0+0 (unknown 3246428776 0 0) 0x7ff91c0559b0 con 0x7ff92410c9d0 2024-02-10T08:12:54.919 INFO:tasks.workunit.client.0.smithi070.stderr:pool '8b5e84ed-f339-493c-81c7-098f848f87fd' already exists 2024-02-10T08:12:54.921 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.920+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 >> v1:172.21.15.70:6800/4101821536 conn(0x7ff91006e980 legacy=0x7ff910070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:54.921 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.920+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 >> v1:172.21.15.70:6789/0 conn(0x7ff92410c9d0 legacy=0x7ff9241a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:54.921 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.920+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 shutdown_connections 2024-02-10T08:12:54.921 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.920+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 >> 172.21.15.70:0/3657439572 conn(0x7ff924076c40 msgr2=0x7ff924068b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:54.921 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.920+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 shutdown_connections 2024-02-10T08:12:54.921 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:54.920+0000 7ff928a2e640 1 -- 172.21.15.70:0/3657439572 wait complete. 2024-02-10T08:12:54.930 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool application enable 8b5e84ed-f339-493c-81c7-098f848f87fd rados 2024-02-10T08:12:55.014 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.011+0000 7fde034c9640 1 -- 172.21.15.70:0/3491548605 >> v1:172.21.15.70:6790/0 conn(0x7fddfc10a050 legacy=0x7fddfc10c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:55.014 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.011+0000 7fde034c9640 1 -- 172.21.15.70:0/3491548605 shutdown_connections 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.011+0000 7fde034c9640 1 -- 172.21.15.70:0/3491548605 >> 172.21.15.70:0/3491548605 conn(0x7fddfc0fba50 msgr2=0x7fddfc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.011+0000 7fde034c9640 1 -- 172.21.15.70:0/3491548605 shutdown_connections 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.011+0000 7fde034c9640 1 -- 172.21.15.70:0/3491548605 wait complete. 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fde034c9640 1 Processor -- start 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fde034c9640 1 -- start start 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fde034c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddfc1a5ef0 con 0x7fddfc10a050 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fde034c9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddfc1a70f0 con 0x7fddfc10db60 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fde034c9640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddfc1a82f0 con 0x7fddfc102250 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fde01265640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7fddfc102250 0x7fddfc109770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:48338/0 (socket says 172.21.15.70:48338) 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fde01265640 1 -- 172.21.15.70:0/706490476 learned_addr learned my addr 172.21.15.70:0/706490476 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306385393 0 0) 0x7fddfc1a5ef0 con 0x7fddfc10a050 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde8003520 con 0x7fddfc10a050 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254215692 0 0) 0x7fddfc1a82f0 con 0x7fddfc102250 2024-02-10T08:12:55.015 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.012+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddfc1a5ef0 con 0x7fddfc102250 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221364949 0 0) 0x7fddfc1a70f0 con 0x7fddfc10db60 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddfc1a82f0 con 0x7fddfc10db60 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2566422310 0 0) 0x7fddfc1a5ef0 con 0x7fddfc102250 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddfc1a70f0 con 0x7fddfc102250 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4262343540 0 0) 0x7fdde8003520 con 0x7fddfc10a050 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddfc1a5ef0 con 0x7fddfc10a050 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fddf80018b0 con 0x7fddfc102250 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fddec0042e0 con 0x7fddfc10a050 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1876646446 0 0) 0x7fddfc1a70f0 con 0x7fddfc102250 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 >> v1:172.21.15.155:6789/0 conn(0x7fddfc10db60 legacy=0x7fddfc1a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 >> v1:172.21.15.70:6789/0 conn(0x7fddfc10a050 legacy=0x7fddfc1a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc1a94f0 con 0x7fddfc102250 2024-02-10T08:12:55.016 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc1a84e0 con 0x7fddfc102250 2024-02-10T08:12:55.017 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc1a8970 con 0x7fddfc102250 2024-02-10T08:12:55.017 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7fddf8001aa0 con 0x7fddfc102250 2024-02-10T08:12:55.017 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.013+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7fddf8005c10 con 0x7fddfc102250 2024-02-10T08:12:55.017 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.014+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde0005080 con 0x7fddfc102250 2024-02-10T08:12:55.018 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.014+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7fddf80038c0 con 0x7fddfc102250 2024-02-10T08:12:55.018 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.015+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(828..828 src has 252..828) v4 ==== 8193+0+0 (unknown 2916237442 0 0) 0x7fddf8088ff0 con 0x7fddfc102250 2024-02-10T08:12:55.020 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.019+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7fddf8055890 con 0x7fddfc102250 2024-02-10T08:12:55.195 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.195+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"} v 0) v1 -- 0x7fdde0005330 con 0x7fddfc102250 2024-02-10T08:12:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:54 smithi070 ceph-mon[39923]: from='client.58739 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:12:55.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:54 smithi070 ceph-mon[39923]: osdmap e827: 8 total, 8 up, 8 in 2024-02-10T08:12:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:54 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]: dispatch 2024-02-10T08:12:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:54 smithi070 ceph-mon[29548]: from='client.58739 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:12:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:54 smithi070 ceph-mon[29548]: osdmap e827: 8 total, 8 up, 8 in 2024-02-10T08:12:55.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:54 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]: dispatch 2024-02-10T08:12:55.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:54 smithi155 ceph-mon[32688]: from='client.58739 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:12:55.321 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:54 smithi155 ceph-mon[32688]: osdmap e827: 8 total, 8 up, 8 in 2024-02-10T08:12:55.321 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:54 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]: dispatch 2024-02-10T08:12:55.818 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.817+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]=0 enabled application 'rados' on pool '8b5e84ed-f339-493c-81c7-098f848f87fd' v829) v1 ==== 213+0+0 (unknown 2568831619 0 0) 0x7fddf8001d40 con 0x7fddfc102250 2024-02-10T08:12:55.932 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:55.931+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"} v 0) v1 -- 0x7fdde0001090 con 0x7fddfc102250 2024-02-10T08:12:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[29548]: pgmap v1785: 176 pgs: 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:12:56.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]': finished 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[29548]: osdmap e828: 8 total, 8 up, 8 in 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]: dispatch 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/706490476' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[29548]: from='client.58760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[39923]: pgmap v1785: 176 pgs: 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]': finished 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[39923]: osdmap e828: 8 total, 8 up, 8 in 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]: dispatch 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/706490476' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:56.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:55 smithi070 ceph-mon[39923]: from='client.58760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:55 smithi155 ceph-mon[32688]: pgmap v1785: 176 pgs: 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:12:56.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]': finished 2024-02-10T08:12:56.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:55 smithi155 ceph-mon[32688]: osdmap e828: 8 total, 8 up, 8 in 2024-02-10T08:12:56.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3657439572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pg_num": 12}]: dispatch 2024-02-10T08:12:56.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:55 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/706490476' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:56.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:55 smithi155 ceph-mon[32688]: from='client.58760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:56.824 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.822+0000 7fddf27fc640 1 -- 172.21.15.70:0/706490476 <== mon.2 v1:172.21.15.70:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]=0 enabled application 'rados' on pool '8b5e84ed-f339-493c-81c7-098f848f87fd' v830) v1 ==== 213+0+0 (unknown 3615447465 0 0) 0x7fddf8053e90 con 0x7fddfc102250 2024-02-10T08:12:56.824 INFO:tasks.workunit.client.0.smithi070.stderr:enabled application 'rados' on pool '8b5e84ed-f339-493c-81c7-098f848f87fd' 2024-02-10T08:12:56.845 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.825+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 >> v1:172.21.15.70:6800/4101821536 conn(0x7fdde806e940 legacy=0x7fdde8070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:56.845 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.825+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 >> v1:172.21.15.70:6790/0 conn(0x7fddfc102250 legacy=0x7fddfc109770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:56.845 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.825+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 shutdown_connections 2024-02-10T08:12:56.846 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.825+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 >> 172.21.15.70:0/706490476 conn(0x7fddfc0fba50 msgr2=0x7fddfc10ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:56.846 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.825+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 shutdown_connections 2024-02-10T08:12:56.846 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.825+0000 7fde034c9640 1 -- 172.21.15.70:0/706490476 wait complete. 2024-02-10T08:12:56.846 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 8b5e84ed-f339-493c-81c7-098f848f87fd max_objects 10 2024-02-10T08:12:56.924 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.922+0000 7f5989f48640 1 -- 172.21.15.70:0/1502276830 >> v1:172.21.15.70:6789/0 conn(0x7f59841001d0 legacy=0x7f59841005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:56.924 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.922+0000 7f5989f48640 1 -- 172.21.15.70:0/1502276830 shutdown_connections 2024-02-10T08:12:56.924 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.922+0000 7f5989f48640 1 -- 172.21.15.70:0/1502276830 >> 172.21.15.70:0/1502276830 conn(0x7f59840fba30 msgr2=0x7f59840fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:56.924 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.922+0000 7f5989f48640 1 -- 172.21.15.70:0/1502276830 shutdown_connections 2024-02-10T08:12:56.924 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.922+0000 7f5989f48640 1 -- 172.21.15.70:0/1502276830 wait complete. 2024-02-10T08:12:56.926 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5989f48640 1 Processor -- start 2024-02-10T08:12:56.926 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5989f48640 1 -- start start 2024-02-10T08:12:56.926 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5989f48640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59841a5e70 con 0x7f598410c9c0 2024-02-10T08:12:56.926 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5989f48640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59841a7070 con 0x7f59841001d0 2024-02-10T08:12:56.926 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5989f48640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59841a8270 con 0x7f5984108e50 2024-02-10T08:12:56.926 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5982ffd640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f5984108e50 0x7f59841a0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:43952/0 (socket says 172.21.15.70:43952) 2024-02-10T08:12:56.926 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5982ffd640 1 -- 172.21.15.70:0/2900837139 learned_addr learned my addr 172.21.15.70:0/2900837139 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.923+0000 7f5983fff640 1 --1- 172.21.15.70:0/2900837139 >> v1:172.21.15.70:6789/0 conn(0x7f598410c9c0 0x7f59841a4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39586/0 (socket says 172.21.15.70:39586) 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204471237 0 0) 0x7f59841a8270 con 0x7f5984108e50 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f596c003160 con 0x7f5984108e50 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473539879 0 0) 0x7f59841a5e70 con 0x7f598410c9c0 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59841a8270 con 0x7f598410c9c0 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774379540 0 0) 0x7f59841a7070 con 0x7f59841001d0 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59841a5e70 con 0x7f59841001d0 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2884837499 0 0) 0x7f596c003160 con 0x7f5984108e50 2024-02-10T08:12:56.927 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59841a7070 con 0x7f5984108e50 2024-02-10T08:12:56.928 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f59780018d0 con 0x7f5984108e50 2024-02-10T08:12:56.928 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 271950501 0 0) 0x7f59841a8270 con 0x7f598410c9c0 2024-02-10T08:12:56.929 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f596c003160 con 0x7f598410c9c0 2024-02-10T08:12:56.929 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f597c001c50 con 0x7f598410c9c0 2024-02-10T08:12:56.929 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2268696025 0 0) 0x7f59841a5e70 con 0x7f59841001d0 2024-02-10T08:12:56.929 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59841a8270 con 0x7f59841001d0 2024-02-10T08:12:56.929 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f5974001360 con 0x7f59841001d0 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2750730532 0 0) 0x7f59841a7070 con 0x7f5984108e50 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 >> v1:172.21.15.155:6789/0 conn(0x7f59841001d0 legacy=0x7f5984069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 >> v1:172.21.15.70:6789/0 conn(0x7f598410c9c0 legacy=0x7f59841a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59841a9470 con 0x7f5984108e50 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59841a8460 con 0x7f5984108e50 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f59841a88f0 con 0x7f5984108e50 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f5978001c60 con 0x7f5984108e50 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.924+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f5978005c60 con 0x7f5984108e50 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.925+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5970005080 con 0x7f5984108e50 2024-02-10T08:12:56.930 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.926+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f59780038c0 con 0x7f5984108e50 2024-02-10T08:12:56.931 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.926+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 8 ==== osd_map(830..830 src has 252..830) v4 ==== 8206+0+0 (unknown 3975574881 0 0) 0x7f5978089100 con 0x7f5984108e50 2024-02-10T08:12:56.931 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:56.931+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f5978055c20 con 0x7f5984108e50 2024-02-10T08:12:57.105 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:57.104+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f5970005330 con 0x7f5984108e50 2024-02-10T08:12:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[39923]: from='client.58760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]': finished 2024-02-10T08:12:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[39923]: osdmap e829: 8 total, 8 up, 8 in 2024-02-10T08:12:57.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/706490476' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:57.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[39923]: from='client.58760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:57.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:57.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:57.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[29548]: from='client.58760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]': finished 2024-02-10T08:12:57.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[29548]: osdmap e829: 8 total, 8 up, 8 in 2024-02-10T08:12:57.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/706490476' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:57.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[29548]: from='client.58760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:57.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:57.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:56 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:57.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:56 smithi155 ceph-mon[32688]: from='client.58760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]': finished 2024-02-10T08:12:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:56 smithi155 ceph-mon[32688]: osdmap e829: 8 total, 8 up, 8 in 2024-02-10T08:12:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:56 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/706490476' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:56 smithi155 ceph-mon[32688]: from='client.58760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]: dispatch 2024-02-10T08:12:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:12:57.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:56 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:12:57.827 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:57.826+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 8b5e84ed-f339-493c-81c7-098f848f87fd v831) v1 ==== 223+0+0 (unknown 2653047368 0 0) 0x7f5978053f90 con 0x7f5984108e50 2024-02-10T08:12:57.935 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:57.933+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 --> v1:172.21.15.70:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f5970001090 con 0x7f5984108e50 2024-02-10T08:12:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[39923]: pgmap v1788: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:12:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[39923]: from='client.58760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]': finished 2024-02-10T08:12:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[39923]: osdmap e830: 8 total, 8 up, 8 in 2024-02-10T08:12:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2900837139' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:58.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[39923]: from='client.58766 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[29548]: pgmap v1788: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:12:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[29548]: from='client.58760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]': finished 2024-02-10T08:12:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[29548]: osdmap e830: 8 total, 8 up, 8 in 2024-02-10T08:12:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2900837139' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:58.243 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:57 smithi070 ceph-mon[29548]: from='client.58766 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:57 smithi155 ceph-mon[32688]: pgmap v1788: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:12:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:57 smithi155 ceph-mon[32688]: from='client.58760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "app": "rados"}]': finished 2024-02-10T08:12:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:57 smithi155 ceph-mon[32688]: osdmap e830: 8 total, 8 up, 8 in 2024-02-10T08:12:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:57 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2900837139' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:58.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:57 smithi155 ceph-mon[32688]: from='client.58766 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:58.833 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:58.832+0000 7f5980ff9640 1 -- 172.21.15.70:0/2900837139 <== mon.2 v1:172.21.15.70:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 8b5e84ed-f339-493c-81c7-098f848f87fd v832) v1 ==== 223+0+0 (unknown 2705702340 0 0) 0x7f59780557b0 con 0x7f5984108e50 2024-02-10T08:12:58.833 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_objects = 10 for pool 8b5e84ed-f339-493c-81c7-098f848f87fd 2024-02-10T08:12:58.836 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:58.835+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 >> v1:172.21.15.70:6800/4101821536 conn(0x7f596c06e9e0 legacy=0x7f596c070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:58.836 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:58.835+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 >> v1:172.21.15.70:6790/0 conn(0x7f5984108e50 legacy=0x7f59841a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:12:58.836 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:58.835+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 shutdown_connections 2024-02-10T08:12:58.837 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:58.835+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 >> 172.21.15.70:0/2900837139 conn(0x7f59840fba30 msgr2=0x7f59840fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:12:58.837 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:58.835+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 shutdown_connections 2024-02-10T08:12:58.837 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:12:58.835+0000 7f5989f48640 1 -- 172.21.15.70:0/2900837139 wait complete. 2024-02-10T08:12:58.847 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 30 2024-02-10T08:12:59.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[29548]: from='client.58766 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[29548]: osdmap e831: 8 total, 8 up, 8 in 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/2900837139' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[29548]: from='client.58766 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[29548]: pgmap v1791: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[39923]: from='client.58766 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[39923]: osdmap e831: 8 total, 8 up, 8 in 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/2900837139' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[39923]: from='client.58766 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:59.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:58 smithi070 ceph-mon[39923]: pgmap v1791: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:12:59.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:58 smithi155 ceph-mon[32688]: from='client.58766 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:12:59.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:58 smithi155 ceph-mon[32688]: osdmap e831: 8 total, 8 up, 8 in 2024-02-10T08:12:59.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:58 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/2900837139' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:59.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:58 smithi155 ceph-mon[32688]: from='client.58766 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:12:59.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:58 smithi155 ceph-mon[32688]: pgmap v1791: 188 pgs: 12 unknown, 176 active+clean; 731 KiB data, 908 MiB used, 714 GiB / 715 GiB avail 2024-02-10T08:13:00.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:59 smithi070 ceph-mon[39923]: from='client.58766 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:13:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:12:59 smithi070 ceph-mon[39923]: osdmap e832: 8 total, 8 up, 8 in 2024-02-10T08:13:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:59 smithi070 ceph-mon[29548]: from='client.58766 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:13:00.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:12:59 smithi070 ceph-mon[29548]: osdmap e832: 8 total, 8 up, 8 in 2024-02-10T08:13:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:59 smithi155 ceph-mon[32688]: from='client.58766 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:13:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:12:59 smithi155 ceph-mon[32688]: osdmap e832: 8 total, 8 up, 8 in 2024-02-10T08:13:00.836 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:13:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:13:01.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:00 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:00 smithi070 ceph-mon[39923]: pgmap v1793: 188 pgs: 188 active+clean; 742 KiB data, 892 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-10T08:13:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:00 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:00 smithi070 ceph-mon[29548]: pgmap v1793: 188 pgs: 188 active+clean; 742 KiB data, 892 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-10T08:13:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:00 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:01.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:00 smithi155 ceph-mon[32688]: pgmap v1793: 188 pgs: 188 active+clean; 742 KiB data, 892 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-10T08:13:02.492 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:13:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:13:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:13:04.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:03 smithi070 ceph-mon[29548]: pgmap v1794: 188 pgs: 188 active+clean; 742 KiB data, 892 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T08:13:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:03 smithi070 ceph-mon[39923]: pgmap v1794: 188 pgs: 188 active+clean; 742 KiB data, 892 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T08:13:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:03 smithi155 ceph-mon[32688]: pgmap v1794: 188 pgs: 188 active+clean; 742 KiB data, 892 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T08:13:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:05 smithi070 ceph-mon[39923]: pgmap v1795: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-02-10T08:13:06.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:05 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:13:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:05 smithi070 ceph-mon[29548]: pgmap v1795: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-02-10T08:13:06.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:05 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:13:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:05 smithi155 ceph-mon[32688]: pgmap v1795: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-02-10T08:13:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:05 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:13:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:06 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:13:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:06 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:13:07.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:06 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:13:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:06 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:13:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:06 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:13:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:06 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:13:07.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:06 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:13:07.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:06 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:13:07.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:06 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:13:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:07 smithi070 ceph-mon[39923]: pgmap v1796: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-10T08:13:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:07 smithi070 ceph-mon[29548]: pgmap v1796: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-10T08:13:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:07 smithi155 ceph-mon[32688]: pgmap v1796: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-10T08:13:10.058 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:09 smithi155 ceph-mon[32688]: pgmap v1797: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T08:13:10.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:09 smithi070 ceph-mon[39923]: pgmap v1797: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T08:13:10.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:09 smithi070 ceph-mon[29548]: pgmap v1797: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T08:13:11.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:13:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:13:12.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:11 smithi070 ceph-mon[39923]: pgmap v1798: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 875 B/s rd, 0 op/s 2024-02-10T08:13:12.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:12.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:12.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:11 smithi070 ceph-mon[29548]: pgmap v1798: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 875 B/s rd, 0 op/s 2024-02-10T08:13:12.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:12.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:11 smithi155 ceph-mon[32688]: pgmap v1798: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 875 B/s rd, 0 op/s 2024-02-10T08:13:12.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:12.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:13:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:13:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:13:14.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:13 smithi070 ceph-mon[29548]: pgmap v1799: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:14.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:13 smithi070 ceph-mon[39923]: pgmap v1799: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:14.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:13 smithi155 ceph-mon[32688]: pgmap v1799: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:15 smithi070 ceph-mon[29548]: pgmap v1800: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:15 smithi070 ceph-mon[39923]: pgmap v1800: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:15 smithi155 ceph-mon[32688]: pgmap v1800: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:18.251 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:17 smithi070 ceph-mon[39923]: pgmap v1801: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:18.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:17 smithi070 ceph-mon[29548]: pgmap v1801: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:18.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:17 smithi155 ceph-mon[32688]: pgmap v1801: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:20.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:19 smithi070 ceph-mon[29548]: pgmap v1802: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:20.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:19 smithi070 ceph-mon[39923]: pgmap v1802: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:19 smithi155 ceph-mon[32688]: pgmap v1802: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:21.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:13:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:13:22.043 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:22.043 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:21 smithi070 ceph-mon[39923]: pgmap v1803: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:22.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:22.043 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:21 smithi070 ceph-mon[29548]: pgmap v1803: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:21 smithi155 ceph-mon[32688]: pgmap v1803: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:22.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:13:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:13:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:13:24.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:23 smithi070 ceph-mon[29548]: pgmap v1804: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:24.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:23 smithi070 ceph-mon[39923]: pgmap v1804: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:24.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:23 smithi155 ceph-mon[32688]: pgmap v1804: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:26.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:25 smithi070 ceph-mon[39923]: pgmap v1805: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:26.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:25 smithi070 ceph-mon[29548]: pgmap v1805: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:25 smithi155 ceph-mon[32688]: pgmap v1805: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:28.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:27 smithi070 ceph-mon[29548]: pgmap v1806: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:28.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:27 smithi070 ceph-mon[39923]: pgmap v1806: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:28.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:27 smithi155 ceph-mon[32688]: pgmap v1806: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:28.848 INFO:tasks.workunit.client.0.smithi070.stderr:++ seq 1 10 2024-02-10T08:13:28.849 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:28.849 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj1 /etc/passwd 2024-02-10T08:13:28.888 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:28.888 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj2 /etc/passwd 2024-02-10T08:13:28.923 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:28.923 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj3 /etc/passwd 2024-02-10T08:13:28.957 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:28.957 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj4 /etc/passwd 2024-02-10T08:13:28.990 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:28.990 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj5 /etc/passwd 2024-02-10T08:13:29.022 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:29.023 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj6 /etc/passwd 2024-02-10T08:13:29.055 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:29.055 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj7 /etc/passwd 2024-02-10T08:13:29.087 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:29.087 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj8 /etc/passwd 2024-02-10T08:13:29.119 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:29.120 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj9 /etc/passwd 2024-02-10T08:13:29.152 INFO:tasks.workunit.client.0.smithi070.stderr:+ for f in `seq 1 10` 2024-02-10T08:13:29.152 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 8b5e84ed-f339-493c-81c7-098f848f87fd put obj10 /etc/passwd 2024-02-10T08:13:29.184 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 30 2024-02-10T08:13:30.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:29 smithi070 ceph-mon[29548]: pgmap v1807: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:30.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:29 smithi070 ceph-mon[39923]: pgmap v1807: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:29 smithi155 ceph-mon[32688]: pgmap v1807: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:31.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:13:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:13:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:31 smithi070 ceph-mon[39923]: pgmap v1808: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:32.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:31 smithi070 ceph-mon[29548]: pgmap v1808: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:32.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:32.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:31 smithi155 ceph-mon[32688]: pgmap v1808: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:13:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:13:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:13:34.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:33 smithi070 ceph-mon[39923]: pgmap v1809: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:34.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:33 smithi070 ceph-mon[29548]: pgmap v1809: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:33 smithi155 ceph-mon[32688]: pgmap v1809: 188 pgs: 188 active+clean; 742 KiB data, 893 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:36.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:35 smithi070 ceph-mon[29548]: pgmap v1810: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:13:36.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:35 smithi070 ceph-mon[39923]: pgmap v1810: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:13:36.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:35 smithi155 ceph-mon[32688]: pgmap v1810: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:13:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[29548]: pgmap v1811: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T08:13:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[29548]: pool '8b5e84ed-f339-493c-81c7-098f848f87fd' is full (reached quota's max_objects: 10) 2024-02-10T08:13:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[29548]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:13:38.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[29548]: osdmap e833: 8 total, 8 up, 8 in 2024-02-10T08:13:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[39923]: pgmap v1811: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T08:13:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[39923]: pool '8b5e84ed-f339-493c-81c7-098f848f87fd' is full (reached quota's max_objects: 10) 2024-02-10T08:13:38.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[39923]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:13:38.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:37 smithi070 ceph-mon[39923]: osdmap e833: 8 total, 8 up, 8 in 2024-02-10T08:13:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:37 smithi155 ceph-mon[32688]: pgmap v1811: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T08:13:38.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:37 smithi155 ceph-mon[32688]: pool '8b5e84ed-f339-493c-81c7-098f848f87fd' is full (reached quota's max_objects: 10) 2024-02-10T08:13:38.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:37 smithi155 ceph-mon[32688]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:13:38.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:37 smithi155 ceph-mon[32688]: osdmap e833: 8 total, 8 up, 8 in 2024-02-10T08:13:40.058 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:39 smithi155 ceph-mon[32688]: pgmap v1813: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:40.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:39 smithi070 ceph-mon[29548]: pgmap v1813: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:40.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:39 smithi070 ceph-mon[39923]: pgmap v1813: 188 pgs: 188 active+clean; 857 KiB data, 897 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:41.006 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:13:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:13:42.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[39923]: pgmap v1814: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:13:42.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[29548]: pgmap v1814: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:13:42.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:41 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:42.243 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:13:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:13:42] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:13:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:41 smithi155 ceph-mon[32688]: pgmap v1814: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:41 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:42.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:13:42.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:41 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:44.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:43 smithi070 ceph-mon[29548]: pgmap v1815: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:44.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:43 smithi070 ceph-mon[39923]: pgmap v1815: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:44.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:43 smithi155 ceph-mon[32688]: pgmap v1815: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:13:46.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:45 smithi070 ceph-mon[29548]: pgmap v1816: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:13:46.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:45 smithi070 ceph-mon[39923]: pgmap v1816: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:13:46.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:45 smithi155 ceph-mon[32688]: pgmap v1816: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:13:48.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:47 smithi070 ceph-mon[29548]: pgmap v1817: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:13:48.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:47 smithi070 ceph-mon[39923]: pgmap v1817: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:13:48.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:47 smithi155 ceph-mon[32688]: pgmap v1817: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:13:50.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:49 smithi070 ceph-mon[39923]: pgmap v1818: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2024-02-10T08:13:50.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:49 smithi070 ceph-mon[29548]: pgmap v1818: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2024-02-10T08:13:50.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:49 smithi155 ceph-mon[32688]: pgmap v1818: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2024-02-10T08:13:51.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:13:50 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:13:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:51 smithi070 ceph-mon[29548]: pgmap v1819: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:52.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:51 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:52.242 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:13:52 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:13:52] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:13:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:51 smithi070 ceph-mon[39923]: pgmap v1819: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:52.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:51 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:51 smithi155 ceph-mon[32688]: pgmap v1819: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:52.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:51 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:13:54.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:53 smithi070 ceph-mon[29548]: pgmap v1820: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:54.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:53 smithi070 ceph-mon[39923]: pgmap v1820: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:54.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:53 smithi155 ceph-mon[32688]: pgmap v1820: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:13:55.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:54 smithi070 ceph-mon[29548]: pgmap v1821: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:55.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:54 smithi070 ceph-mon[39923]: pgmap v1821: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:55.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:54 smithi155 ceph-mon[32688]: pgmap v1821: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:13:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:57 smithi070 ceph-mon[29548]: pgmap v1822: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:57.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:57 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:57 smithi070 ceph-mon[39923]: pgmap v1822: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:57.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:57 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:57 smithi155 ceph-mon[32688]: pgmap v1822: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:13:58.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:57 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:13:59.185 INFO:tasks.workunit.client.0.smithi070.stderr:+ rados -p 939b67e9-3d3f-4cc4-aedd-d4506338adca put threemore /etc/passwd 2024-02-10T08:13:59.225 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 939b67e9-3d3f-4cc4-aedd-d4506338adca max_bytes 0 2024-02-10T08:13:59.315 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- 172.21.15.70:0/1270638243 >> v1:172.21.15.70:6789/0 conn(0x7f72e8109f50 legacy=0x7f72e810c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- 172.21.15.70:0/1270638243 shutdown_connections 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- 172.21.15.70:0/1270638243 >> 172.21.15.70:0/1270638243 conn(0x7f72e80fcb10 msgr2=0x7f72e80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- 172.21.15.70:0/1270638243 shutdown_connections 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- 172.21.15.70:0/1270638243 wait complete. 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 Processor -- start 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- start start 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f72e81a7f50 con 0x7f72e81012d0 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f72e81a9150 con 0x7f72e8109f50 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.312+0000 7f72f053a640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f72e81aa350 con 0x7f72e810dac0 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72ee2d6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72e81012d0 0x7f72e810c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50770/0 (socket says 172.21.15.70:50770) 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72eead7640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f72e810dac0 0x7f72e81a6650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:40376/0 (socket says 172.21.15.70:40376) 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72ee2d6640 1 -- 172.21.15.70:0/3139006901 learned_addr learned my addr 172.21.15.70:0/3139006901 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:13:59.316 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213536632 0 0) 0x7f72e81aa350 con 0x7f72e810dac0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d4003160 con 0x7f72e810dac0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428089553 0 0) 0x7f72e81a7f50 con 0x7f72e81012d0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e81aa350 con 0x7f72e81012d0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3178173034 0 0) 0x7f72e81aa350 con 0x7f72e81012d0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f72e81a7f50 con 0x7f72e81012d0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f72e40019c0 con 0x7f72e81012d0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 885259340 0 0) 0x7f72d4003160 con 0x7f72e810dac0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f72e81aa350 con 0x7f72e810dac0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f72e0001bd0 con 0x7f72e810dac0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047539325 0 0) 0x7f72e81a9150 con 0x7f72e8109f50 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d4003160 con 0x7f72e8109f50 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3632525685 0 0) 0x7f72e81a7f50 con 0x7f72e81012d0 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 >> v1:172.21.15.70:6790/0 conn(0x7f72e810dac0 legacy=0x7f72e81a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:59.317 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 >> v1:172.21.15.155:6789/0 conn(0x7f72e8109f50 legacy=0x7f72e810c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:59.318 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e81ab550 con 0x7f72e81012d0 2024-02-10T08:13:59.318 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e81a8140 con 0x7f72e81012d0 2024-02-10T08:13:59.318 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e81a85d0 con 0x7f72e81012d0 2024-02-10T08:13:59.318 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f72e4001cf0 con 0x7f72e81012d0 2024-02-10T08:13:59.318 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.313+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f72e4005d80 con 0x7f72e81012d0 2024-02-10T08:13:59.318 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.314+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc005080 con 0x7f72e81012d0 2024-02-10T08:13:59.318 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.315+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f72e4003940 con 0x7f72e81012d0 2024-02-10T08:13:59.319 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.315+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(833..833 src has 252..833) v4 ==== 8206+0+0 (unknown 348465660 0 0) 0x7f72e4089270 con 0x7f72e81012d0 2024-02-10T08:13:59.319 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.315+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=834}) v3 -- 0x7f72e81a7f50 con 0x7f72e81012d0 2024-02-10T08:13:59.320 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.320+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f72e4055ae0 con 0x7f72e81012d0 2024-02-10T08:13:59.494 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.493+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f72cc005330 con 0x7f72e81012d0 2024-02-10T08:13:59.770 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.769+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v834) v1 ==== 217+0+0 (unknown 3248478898 0 0) 0x7f72e40540e0 con 0x7f72e81012d0 2024-02-10T08:13:59.770 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.770+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 11 ==== osd_map(834..834 src has 252..834) v4 ==== 628+0+0 (unknown 139646696 0 0) 0x7f72e40566a0 con 0x7f72e81012d0 2024-02-10T08:13:59.771 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.770+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=835}) v3 -- 0x7f72e81a9150 con 0x7f72e81012d0 2024-02-10T08:13:59.881 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:13:59.880+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f72cc000c40 con 0x7f72e81012d0 2024-02-10T08:14:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:59 smithi070 ceph-mon[29548]: pgmap v1823: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:00.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:13:59 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:14:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:59 smithi070 ceph-mon[39923]: pgmap v1823: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:00.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:13:59 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:14:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:59 smithi155 ceph-mon[32688]: pgmap v1823: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:00.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:13:59 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:14:00.786 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.784+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v835) v1 ==== 217+0+0 (unknown 281335676 0 0) 0x7f72e4055900 con 0x7f72e81012d0 2024-02-10T08:14:00.786 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_bytes = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:14:00.786 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.784+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 <== mon.0 v1:172.21.15.70:6789/0 13 ==== osd_map(835..835 src has 252..835) v4 ==== 628+0+0 (unknown 725056615 0 0) 0x7f72e405e740 con 0x7f72e81012d0 2024-02-10T08:14:00.786 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.785+0000 7f72df7fe640 1 -- 172.21.15.70:0/3139006901 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=836}) v3 -- 0x7f72d407c370 con 0x7f72e81012d0 2024-02-10T08:14:00.789 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.788+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 >> v1:172.21.15.70:6800/4101821536 conn(0x7f72d406e990 legacy=0x7f72d4070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:00.789 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.788+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 >> v1:172.21.15.70:6789/0 conn(0x7f72e81012d0 legacy=0x7f72e810c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:00.790 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.788+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 shutdown_connections 2024-02-10T08:14:00.790 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.788+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 >> 172.21.15.70:0/3139006901 conn(0x7f72e80fcb10 msgr2=0x7f72e8107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:00.790 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.788+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 shutdown_connections 2024-02-10T08:14:00.790 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.788+0000 7f72f053a640 1 -- 172.21.15.70:0/3139006901 wait complete. 2024-02-10T08:14:00.804 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool set-quota 939b67e9-3d3f-4cc4-aedd-d4506338adca max_objects 0 2024-02-10T08:14:00.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.889+0000 7f830e284640 1 -- 172.21.15.70:0/2883181665 >> v1:172.21.15.70:6789/0 conn(0x7f8308109f50 legacy=0x7f830810c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:00.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.889+0000 7f830e284640 1 -- 172.21.15.70:0/2883181665 shutdown_connections 2024-02-10T08:14:00.890 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.889+0000 7f830e284640 1 -- 172.21.15.70:0/2883181665 >> 172.21.15.70:0/2883181665 conn(0x7f83080fcb10 msgr2=0x7f83080fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:00.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.889+0000 7f830e284640 1 -- 172.21.15.70:0/2883181665 shutdown_connections 2024-02-10T08:14:00.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.889+0000 7f830e284640 1 -- 172.21.15.70:0/2883181665 wait complete. 2024-02-10T08:14:00.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.889+0000 7f830e284640 1 Processor -- start 2024-02-10T08:14:00.891 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f830e284640 1 -- start start 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f830e284640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f83081a8190 con 0x7f8308109f50 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f830e284640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f83081a9390 con 0x7f830810dac0 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f830e284640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f83081aa590 con 0x7f83081012d0 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f8307fff640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f83081012d0 0x7f8308106570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:40390/0 (socket says 172.21.15.70:40390) 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83077fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8308109f50 0x7f83081a3220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50792/0 (socket says 172.21.15.70:50792) 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f8307fff640 1 -- 172.21.15.70:0/753042732 learned_addr learned my addr 172.21.15.70:0/753042732 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486369462 0 0) 0x7f83081a8190 con 0x7f8308109f50 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f0003160 con 0x7f8308109f50 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022043738 0 0) 0x7f83081aa590 con 0x7f83081012d0 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83081a8190 con 0x7f83081012d0 2024-02-10T08:14:00.894 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229343184 0 0) 0x7f83081a9390 con 0x7f830810dac0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83081aa590 con 0x7f830810dac0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2235473761 0 0) 0x7f82f0003160 con 0x7f8308109f50 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83081a9390 con 0x7f8308109f50 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.890+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f82fc001550 con 0x7f8308109f50 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 878112470 0 0) 0x7f83081a8190 con 0x7f83081012d0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82f0003160 con 0x7f83081012d0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f82f8001800 con 0x7f83081012d0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1601700120 0 0) 0x7f83081aa590 con 0x7f830810dac0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f83081a8190 con 0x7f830810dac0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f8300001af0 con 0x7f830810dac0 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3504422065 0 0) 0x7f83081a9390 con 0x7f8308109f50 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 >> v1:172.21.15.70:6790/0 conn(0x7f83081012d0 legacy=0x7f8308106570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 >> v1:172.21.15.155:6789/0 conn(0x7f830810dac0 legacy=0x7f83081a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83081ab790 con 0x7f8308109f50 2024-02-10T08:14:00.895 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081a8380 con 0x7f8308109f50 2024-02-10T08:14:00.896 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081a8810 con 0x7f8308109f50 2024-02-10T08:14:00.896 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f82fc001130 con 0x7f8308109f50 2024-02-10T08:14:00.896 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.891+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f82fc004da0 con 0x7f8308109f50 2024-02-10T08:14:00.896 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.892+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f4005080 con 0x7f8308109f50 2024-02-10T08:14:00.896 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.892+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f82fc004fe0 con 0x7f8308109f50 2024-02-10T08:14:00.896 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.893+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(835..835 src has 252..835) v4 ==== 8206+0+0 (unknown 1703478248 0 0) 0x7f82fc0892d0 con 0x7f8308109f50 2024-02-10T08:14:00.896 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.893+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=836}) v3 -- 0x7f83081a9390 con 0x7f8308109f50 2024-02-10T08:14:00.898 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:00.897+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f82fc055dd0 con 0x7f8308109f50 2024-02-10T08:14:01.008 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:14:00 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:14:01.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:14:01.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:00 smithi155 ceph-mon[32688]: osdmap e834: 8 total, 8 up, 8 in 2024-02-10T08:14:01.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:00 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:14:01.072 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:01.071+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f82f4005330 con 0x7f8308109f50 2024-02-10T08:14:01.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:14:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:00 smithi070 ceph-mon[29548]: osdmap e834: 8 total, 8 up, 8 in 2024-02-10T08:14:01.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:00 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:14:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:14:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:00 smithi070 ceph-mon[39923]: osdmap e834: 8 total, 8 up, 8 in 2024-02-10T08:14:01.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:00 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:14:01.787 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:01.786+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v836) v1 ==== 221+0+0 (unknown 2815735947 0 0) 0x7f82fc054140 con 0x7f8308109f50 2024-02-10T08:14:01.787 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:01.786+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 11 ==== osd_map(836..836 src has 252..836) v4 ==== 628+0+0 (unknown 2024534363 0 0) 0x7f82fc056700 con 0x7f8308109f50 2024-02-10T08:14:01.787 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:01.786+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=837}) v3 -- 0x7f83081aa590 con 0x7f8308109f50 2024-02-10T08:14:01.899 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:01.898+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f82f4000c40 con 0x7f8308109f50 2024-02-10T08:14:02.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[39923]: pgmap v1825: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:14:02.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[39923]: osdmap e835: 8 total, 8 up, 8 in 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[29548]: pgmap v1825: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[29548]: osdmap e835: 8 total, 8 up, 8 in 2024-02-10T08:14:02.060 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:01 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:14:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:01 smithi155 ceph-mon[32688]: pgmap v1825: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:14:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:01 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3139006901' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:14:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:01 smithi155 ceph-mon[32688]: osdmap e835: 8 total, 8 up, 8 in 2024-02-10T08:14:02.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:01 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:14:02.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:02 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:14:02] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:14:02.800 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.799+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca v837) v1 ==== 221+0+0 (unknown 662989162 0 0) 0x7f82fc055960 con 0x7f8308109f50 2024-02-10T08:14:02.800 INFO:tasks.workunit.client.0.smithi070.stderr:set-quota max_objects = 0 for pool 939b67e9-3d3f-4cc4-aedd-d4506338adca 2024-02-10T08:14:02.800 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.799+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 <== mon.0 v1:172.21.15.70:6789/0 13 ==== osd_map(837..837 src has 252..837) v4 ==== 628+0+0 (unknown 2829546442 0 0) 0x7f82fc05e7a0 con 0x7f8308109f50 2024-02-10T08:14:02.800 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.799+0000 7f83057fa640 1 -- 172.21.15.70:0/753042732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=838}) v3 -- 0x7f82f007c3a0 con 0x7f8308109f50 2024-02-10T08:14:02.805 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.804+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 >> v1:172.21.15.70:6800/4101821536 conn(0x7f82f006e9e0 legacy=0x7f82f0070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:02.806 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.804+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 >> v1:172.21.15.70:6789/0 conn(0x7f8308109f50 legacy=0x7f83081a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:02.806 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.804+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 shutdown_connections 2024-02-10T08:14:02.806 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.804+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 >> 172.21.15.70:0/753042732 conn(0x7f83080fcb10 msgr2=0x7f8308110fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:02.806 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.805+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 shutdown_connections 2024-02-10T08:14:02.806 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:02.805+0000 7f830e284640 1 -- 172.21.15.70:0/753042732 wait complete. 2024-02-10T08:14:02.819 INFO:tasks.workunit.client.0.smithi070.stderr:+ sleep 30 2024-02-10T08:14:03.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:14:03.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:02 smithi070 ceph-mon[29548]: osdmap e836: 8 total, 8 up, 8 in 2024-02-10T08:14:03.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:02 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:14:03.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:14:03.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:02 smithi070 ceph-mon[39923]: osdmap e836: 8 total, 8 up, 8 in 2024-02-10T08:14:03.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:02 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:14:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:14:03.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:02 smithi155 ceph-mon[32688]: osdmap e836: 8 total, 8 up, 8 in 2024-02-10T08:14:03.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:02 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:14:04.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:03 smithi070 ceph-mon[29548]: pgmap v1828: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:04.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:03 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:14:04.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:03 smithi070 ceph-mon[29548]: osdmap e837: 8 total, 8 up, 8 in 2024-02-10T08:14:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:03 smithi070 ceph-mon[39923]: pgmap v1828: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:03 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:14:04.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:03 smithi070 ceph-mon[39923]: osdmap e837: 8 total, 8 up, 8 in 2024-02-10T08:14:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:03 smithi155 ceph-mon[32688]: pgmap v1828: 188 pgs: 188 active+clean; 857 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:03 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/753042732' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:14:04.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:03 smithi155 ceph-mon[32688]: osdmap e837: 8 total, 8 up, 8 in 2024-02-10T08:14:06.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:05 smithi070 ceph-mon[29548]: pgmap v1830: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-10T08:14:06.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:05 smithi070 ceph-mon[39923]: pgmap v1830: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-10T08:14:06.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:05 smithi155 ceph-mon[32688]: pgmap v1830: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-10T08:14:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:06 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:06 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:07.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:06 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[29548]: pgmap v1831: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[39923]: pgmap v1831: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:14:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:14:08.243 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:07 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:07 smithi155 ceph-mon[32688]: pgmap v1831: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-10T08:14:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:07 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:07 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:07 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:14:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:07 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:14:08.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:07 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' 2024-02-10T08:14:10.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:09 smithi070 ceph-mon[29548]: pgmap v1832: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 659 B/s rd, 1.5 KiB/s wr, 0 op/s 2024-02-10T08:14:10.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:09 smithi070 ceph-mon[39923]: pgmap v1832: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 659 B/s rd, 1.5 KiB/s wr, 0 op/s 2024-02-10T08:14:10.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:09 smithi155 ceph-mon[32688]: pgmap v1832: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 659 B/s rd, 1.5 KiB/s wr, 0 op/s 2024-02-10T08:14:11.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:14:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:14:12.241 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:12 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:14:12] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:14:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:11 smithi070 ceph-mon[29548]: pgmap v1833: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-10T08:14:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:11 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:12.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:11 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:14:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:11 smithi070 ceph-mon[39923]: pgmap v1833: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-10T08:14:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:11 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:12.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:11 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:14:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:11 smithi155 ceph-mon[32688]: pgmap v1833: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-10T08:14:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:11 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:12.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:11 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:14:14.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:13 smithi070 ceph-mon[29548]: pgmap v1834: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T08:14:14.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:13 smithi070 ceph-mon[39923]: pgmap v1834: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T08:14:14.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:13 smithi155 ceph-mon[32688]: pgmap v1834: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-10T08:14:16.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:15 smithi070 ceph-mon[29548]: pgmap v1835: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-10T08:14:16.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:15 smithi070 ceph-mon[39923]: pgmap v1835: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-10T08:14:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:15 smithi155 ceph-mon[32688]: pgmap v1835: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-02-10T08:14:18.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:17 smithi070 ceph-mon[29548]: pgmap v1836: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:18.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:17 smithi070 ceph-mon[39923]: pgmap v1836: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:18.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:17 smithi155 ceph-mon[32688]: pgmap v1836: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:20.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:19 smithi070 ceph-mon[29548]: pgmap v1837: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:20.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:19 smithi070 ceph-mon[39923]: pgmap v1837: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:20.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:19 smithi155 ceph-mon[32688]: pgmap v1837: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:21.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:14:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:14:22.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:21 smithi070 ceph-mon[29548]: pgmap v1838: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:22.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:21 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:22.242 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:22 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:14:22] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:14:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:21 smithi070 ceph-mon[39923]: pgmap v1838: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:22.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:21 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:21 smithi155 ceph-mon[32688]: pgmap v1838: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:22.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:21 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:24.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:23 smithi070 ceph-mon[29548]: pgmap v1839: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:24.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:23 smithi070 ceph-mon[39923]: pgmap v1839: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:24.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:23 smithi155 ceph-mon[32688]: pgmap v1839: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:26.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:25 smithi070 ceph-mon[29548]: pgmap v1840: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:26.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:25 smithi070 ceph-mon[39923]: pgmap v1840: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:26.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:25 smithi155 ceph-mon[32688]: pgmap v1840: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:27.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:26 smithi070 ceph-mon[29548]: pgmap v1841: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:27.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:26 smithi070 ceph-mon[29548]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:14:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:26 smithi070 ceph-mon[39923]: pgmap v1841: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:27.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:26 smithi070 ceph-mon[39923]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:14:27.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:26 smithi155 ceph-mon[32688]: pgmap v1841: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:27.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:26 smithi155 ceph-mon[32688]: from='mgr.34502 v1:172.21.15.70:0/2987828597' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T08:14:30.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:29 smithi070 ceph-mon[39923]: pgmap v1842: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:30.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:29 smithi070 ceph-mon[29548]: pgmap v1842: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:30.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:29 smithi155 ceph-mon[32688]: pgmap v1842: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:31.005 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:14:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug there is no tcmu-runner data available 2024-02-10T08:14:32.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:31 smithi070 ceph-mon[29548]: pgmap v1843: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:32.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:31 smithi070 ceph-mon[29548]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:31 smithi070 ceph-mon[39923]: pgmap v1843: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:32.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:31 smithi070 ceph-mon[39923]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:31 smithi155 ceph-mon[32688]: pgmap v1843: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:14:32.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:31 smithi155 ceph-mon[32688]: from='client.24466 v1:172.21.15.155:0/3378300928' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:14:32.491 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:32 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: ::ffff:172.21.15.155 - - [10/Feb/2024:08:14:32] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T08:14:32.820 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool delete 939b67e9-3d3f-4cc4-aedd-d4506338adca 939b67e9-3d3f-4cc4-aedd-d4506338adca --yes-i-really-really-mean-it 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.911+0000 7f4b64313640 1 -- 172.21.15.70:0/616721453 >> v1:172.21.15.70:6790/0 conn(0x7f4b5c1012e0 legacy=0x7f4b5c1016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- 172.21.15.70:0/616721453 shutdown_connections 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- 172.21.15.70:0/616721453 >> 172.21.15.70:0/616721453 conn(0x7f4b5c0fcb60 msgr2=0x7f4b5c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- 172.21.15.70:0/616721453 shutdown_connections 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- 172.21.15.70:0/616721453 wait complete. 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 Processor -- start 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- start start 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b5c1a5fa0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.916 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b5c1a7160 con 0x7f4b5c10dad0 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.912+0000 7f4b64313640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b5c1a8360 con 0x7f4b5c109f60 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b620af640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4b5c1012e0 0x7f4b5c106590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58476/0 (socket says 172.21.15.70:58476) 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b620af640 1 -- 172.21.15.70:0/3126833340 learned_addr learned my addr 172.21.15.70:0/3126833340 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386873197 0 0) 0x7f4b5c1a5fa0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b48003520 con 0x7f4b5c1012e0 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654461832 0 0) 0x7f4b5c1a8360 con 0x7f4b5c109f60 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b5c1a5fa0 con 0x7f4b5c109f60 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3066411185 0 0) 0x7f4b48003520 con 0x7f4b5c1012e0 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b5c1a8360 con 0x7f4b5c1012e0 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2705388880 0 0) 0x7f4b5c1a5fa0 con 0x7f4b5c109f60 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b48003520 con 0x7f4b5c109f60 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4b580015d0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4b4c001830 con 0x7f4b5c109f60 2024-02-10T08:14:32.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634635077 0 0) 0x7f4b5c1a7160 con 0x7f4b5c10dad0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b5c1a5fa0 con 0x7f4b5c10dad0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 113975740 0 0) 0x7f4b5c1a8360 con 0x7f4b5c1012e0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 >> v1:172.21.15.70:6790/0 conn(0x7f4b5c109f60 legacy=0x7f4b5c1a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 >> v1:172.21.15.155:6789/0 conn(0x7f4b5c10dad0 legacy=0x7f4b5c1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c1a9560 con 0x7f4b5c1012e0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c1a8550 con 0x7f4b5c1012e0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.913+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c1a89e0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.914+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f4b580011b0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.914+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f4b58004eb0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.915+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b40005080 con 0x7f4b5c1012e0 2024-02-10T08:14:32.918 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.915+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f4b580050f0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.919 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.916+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(837..837 src has 252..837) v4 ==== 8206+0+0 (unknown 4133322196 0 0) 0x7f4b580892f0 con 0x7f4b5c1012e0 2024-02-10T08:14:32.919 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.916+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=838}) v3 -- 0x7f4b5c1a8360 con 0x7f4b5c1012e0 2024-02-10T08:14:32.920 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:32.920+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f4b58055e10 con 0x7f4b5c1012e0 2024-02-10T08:14:33.095 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.094+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b40005330 con 0x7f4b5c1012e0 2024-02-10T08:14:33.789 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.788+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]=0 pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' removed v838) v1 ==== 248+0+0 (unknown 3605979788 0 0) 0x7f4b58054180 con 0x7f4b5c1012e0 2024-02-10T08:14:33.789 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.788+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 11 ==== osd_map(838..838 src has 252..838) v4 ==== 296+0+0 (unknown 3947887558 0 0) 0x7f4b5805a190 con 0x7f4b5c1012e0 2024-02-10T08:14:33.789 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.788+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=839}) v3 -- 0x7f4b48003520 con 0x7f4b5c1012e0 2024-02-10T08:14:33.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.904+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b40005d60 con 0x7f4b5c1012e0 2024-02-10T08:14:33.905 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.904+0000 7f4b537fe640 1 -- 172.21.15.70:0/3126833340 <== mon.0 v1:172.21.15.70:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]=0 pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' does not exist v838) v1 ==== 255+0+0 (unknown 3645511942 0 0) 0x7f4b580559a0 con 0x7f4b5c1012e0 2024-02-10T08:14:33.906 INFO:tasks.workunit.client.0.smithi070.stderr:pool '939b67e9-3d3f-4cc4-aedd-d4506338adca' does not exist 2024-02-10T08:14:33.907 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.906+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 >> v1:172.21.15.70:6800/4101821536 conn(0x7f4b4806e920 legacy=0x7f4b48070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:33.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.906+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 >> v1:172.21.15.70:6789/0 conn(0x7f4b5c1012e0 legacy=0x7f4b5c106590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:33.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.906+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 shutdown_connections 2024-02-10T08:14:33.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.906+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 >> 172.21.15.70:0/3126833340 conn(0x7f4b5c0fcb60 msgr2=0x7f4b5c110760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:33.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.906+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 shutdown_connections 2024-02-10T08:14:33.908 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.906+0000 7f4b64313640 1 -- 172.21.15.70:0/3126833340 wait complete. 2024-02-10T08:14:33.916 INFO:tasks.workunit.client.0.smithi070.stderr:+ ceph osd pool delete 8b5e84ed-f339-493c-81c7-098f848f87fd 8b5e84ed-f339-493c-81c7-098f848f87fd --yes-i-really-really-mean-it 2024-02-10T08:14:34.002 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.997+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1388661422 >> v1:172.21.15.70:6790/0 conn(0x7f98dc0ffa90 legacy=0x7f98dc0ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:34.002 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.997+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1388661422 shutdown_connections 2024-02-10T08:14:34.002 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.997+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1388661422 >> 172.21.15.70:0/1388661422 conn(0x7f98dc0752b0 msgr2=0x7f98dc0756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1388661422 shutdown_connections 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1388661422 wait complete. 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e3e0e640 1 Processor -- start 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e3e0e640 1 -- start start 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e3e0e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f98dc1a8200 con 0x7f98dc109f80 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e3e0e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f98dc1a9400 con 0x7f98dc0ffa90 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e3e0e640 1 -- --> v1:172.21.15.70:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f98dc1aa600 con 0x7f98dc10daf0 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e13a9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f98dc109f80 0x7f98dc1a3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58504/0 (socket says 172.21.15.70:58504) 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e23ab640 1 --1- >> v1:172.21.15.70:6790/0 conn(0x7f98dc10daf0 0x7f98dc1a6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.70:6790/0 says I am v1:172.21.15.70:33378/0 (socket says 172.21.15.70:33378) 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.998+0000 7f98e13a9640 1 -- 172.21.15.70:0/1313505136 learned_addr learned my addr 172.21.15.70:0/1313505136 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214158821 0 0) 0x7f98dc1a8200 con 0x7f98dc109f80 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c8003520 con 0x7f98dc109f80 2024-02-10T08:14:34.003 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.2 v1:172.21.15.70:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174912576 0 0) 0x7f98dc1aa600 con 0x7f98dc10daf0 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98dc1a8200 con 0x7f98dc10daf0 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404077126 0 0) 0x7f98dc1a9400 con 0x7f98dc0ffa90 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98dc1aa600 con 0x7f98dc0ffa90 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3657429964 0 0) 0x7f98c8003520 con 0x7f98dc109f80 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f98dc1a9400 con 0x7f98dc109f80 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f98cc0019a0 con 0x7f98dc109f80 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.2 v1:172.21.15.70:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3302804831 0 0) 0x7f98dc1a8200 con 0x7f98dc10daf0 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f98c8003520 con 0x7f98dc10daf0 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.2 v1:172.21.15.70:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f98d4001b70 con 0x7f98dc10daf0 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 198012002 0 0) 0x7f98dc1a9400 con 0x7f98dc109f80 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 >> v1:172.21.15.70:6790/0 conn(0x7f98dc10daf0 legacy=0x7f98dc1a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 >> v1:172.21.15.155:6789/0 conn(0x7f98dc0ffa90 legacy=0x7f98dc1065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:34.004 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc1ab800 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc1aa7f0 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc1aac80 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 326295009 0 0) 0x7f98cc003a80 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:33.999+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 264298066 0 0) 0x7f98cc005ba0 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.000+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0005080 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.000+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 7 ==== mgrmap(e 21) v1 ==== 91428+0+0 (unknown 1934439254 0 0) 0x7f98cc0035f0 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.001+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 8 ==== osd_map(838..838 src has 252..838) v4 ==== 7818+0+0 (unknown 3362917696 0 0) 0x7f98cc088eb0 con 0x7f98dc109f80 2024-02-10T08:14:34.005 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.001+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=839}) v3 -- 0x7f98dc1a9400 con 0x7f98dc109f80 2024-02-10T08:14:34.006 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.005+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (unknown 1092875540 0 1883366685) 0x7f98cc055b40 con 0x7f98dc109f80 2024-02-10T08:14:34.180 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.179+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pool2": "8b5e84ed-f339-493c-81c7-098f848f87fd", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98c0005330 con 0x7f98dc109f80 2024-02-10T08:14:34.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:33 smithi070 ceph-mon[29548]: pgmap v1844: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:34.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:33 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:33 smithi070 ceph-mon[39923]: pgmap v1844: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:34.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:33 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:33 smithi155 ceph-mon[32688]: pgmap v1844: 188 pgs: 188 active+clean; 868 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:14:34.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:33 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:34.803 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.802+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pool2": "8b5e84ed-f339-493c-81c7-098f848f87fd", "yes_i_really_really_mean_it": true}]=0 pool '8b5e84ed-f339-493c-81c7-098f848f87fd' removed v839) v1 ==== 248+0+0 (unknown 3768966290 0 0) 0x7f98cc053eb0 con 0x7f98dc109f80 2024-02-10T08:14:34.804 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.804+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 11 ==== osd_map(839..839 src has 252..839) v4 ==== 296+0+0 (unknown 2457679447 0 0) 0x7f98cc056470 con 0x7f98dc109f80 2024-02-10T08:14:34.804 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.804+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_subscribe({osdmap=840}) v3 -- 0x7f98dc1a8200 con 0x7f98dc109f80 2024-02-10T08:14:34.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.916+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 --> v1:172.21.15.70:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pool2": "8b5e84ed-f339-493c-81c7-098f848f87fd", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98c0001180 con 0x7f98dc109f80 2024-02-10T08:14:34.917 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.916+0000 7f98d2ffd640 1 -- 172.21.15.70:0/1313505136 <== mon.0 v1:172.21.15.70:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pool2": "8b5e84ed-f339-493c-81c7-098f848f87fd", "yes_i_really_really_mean_it": true}]=0 pool '8b5e84ed-f339-493c-81c7-098f848f87fd' does not exist v839) v1 ==== 255+0+0 (unknown 1762027657 0 0) 0x7f98cc0556d0 con 0x7f98dc109f80 2024-02-10T08:14:34.917 INFO:tasks.workunit.client.0.smithi070.stderr:pool '8b5e84ed-f339-493c-81c7-098f848f87fd' does not exist 2024-02-10T08:14:34.919 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.918+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 >> v1:172.21.15.70:6800/4101821536 conn(0x7f98c806e970 legacy=0x7f98c8070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:34.919 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.918+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 >> v1:172.21.15.70:6789/0 conn(0x7f98dc109f80 legacy=0x7f98dc1a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:34.920 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.918+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 shutdown_connections 2024-02-10T08:14:34.920 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.918+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 >> 172.21.15.70:0/1313505136 conn(0x7f98dc0752b0 msgr2=0x7f98dc111220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:34.920 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.918+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 shutdown_connections 2024-02-10T08:14:34.920 INFO:tasks.workunit.client.0.smithi070.stderr:2024-02-10T08:14:34.918+0000 7f98e3e0e640 1 -- 172.21.15.70:0/1313505136 wait complete. 2024-02-10T08:14:34.928 INFO:tasks.workunit.client.0.smithi070.stderr:+ echo OK 2024-02-10T08:14:34.929 INFO:tasks.workunit.client.0.smithi070.stdout:OK 2024-02-10T08:14:34.930 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-10T08:14:34.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-10T08:14:34.959 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-10T08:14:34.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-10T08:14:35.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:14:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[29548]: osdmap e838: 8 total, 8 up, 8 in 2024-02-10T08:14:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:35.242 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[29548]: from='client.? v1:172.21.15.70:0/1313505136' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pool2": "8b5e84ed-f339-493c-81c7-098f848f87fd", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:35.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:14:35.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[39923]: osdmap e838: 8 total, 8 up, 8 in 2024-02-10T08:14:35.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:35.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:34 smithi070 ceph-mon[39923]: from='client.? v1:172.21.15.70:0/1313505136' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pool2": "8b5e84ed-f339-493c-81c7-098f848f87fd", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:14:35.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:34 smithi155 ceph-mon[32688]: osdmap e838: 8 total, 8 up, 8 in 2024-02-10T08:14:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/3126833340' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "pool2": "939b67e9-3d3f-4cc4-aedd-d4506338adca", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:35.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:34 smithi155 ceph-mon[32688]: from='client.? v1:172.21.15.70:0/1313505136' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8b5e84ed-f339-493c-81c7-098f848f87fd", "pool2": "8b5e84ed-f339-493c-81c7-098f848f87fd", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:14:35.265 DEBUG:teuthology.parallel:result is None 2024-02-10T08:14:35.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T08:14:35.292 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T08:14:35.292 DEBUG:teuthology.orchestra.run.smithi070:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-10T08:14:35.354 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T08:14:35.354 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-10T08:14:35.364 INFO:tasks.cephadm:Teardown begin 2024-02-10T08:14:35.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:14:35.421 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:14:35.456 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-10T08:14:35.456 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T08:14:35.475 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T08:14:35.514 INFO:tasks.cephadm:Stopping all daemons... 2024-02-10T08:14:35.515 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-10T08:14:35.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a 2024-02-10T08:14:35.792 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:35 smithi070 systemd[1]: Stopping Ceph mon.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:36.127 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a[29544]: 2024-02-10T08:14:35.791+0000 7fe37cb6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:14:36.127 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a[29544]: 2024-02-10T08:14:35.791+0000 7fe37cb6b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-10T08:14:36.127 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:35 smithi070 podman[155607]: 2024-02-10 08:14:35.864308052 +0000 UTC m=+0.171301888 container died b699ece04241846664645dc1675f01f2d7258650257e7d96b3e587b2b5bd9002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, RELEASE=main-290212a, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-10T08:14:36.127 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:35] ENGINE Bus STOPPING 2024-02-10T08:14:36.127 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:35] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T08:14:36.127 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:35] ENGINE Bus STOPPED 2024-02-10T08:14:36.128 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:35] ENGINE Bus STARTING 2024-02-10T08:14:36.128 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:35] ENGINE Serving on http://:::9283 2024-02-10T08:14:36.128 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:35 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:35] ENGINE Bus STARTED 2024-02-10T08:14:36.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:36 smithi070 podman[155607]: 2024-02-10 08:14:36.301005716 +0000 UTC m=+0.607999551 container cleanup b699ece04241846664645dc1675f01f2d7258650257e7d96b3e587b2b5bd9002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a) 2024-02-10T08:14:36.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:36 smithi070 bash[155607]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a 2024-02-10T08:14:36.770 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:36 smithi070 podman[155632]: 2024-02-10 08:14:36.519414143 +0000 UTC m=+0.655423693 container remove b699ece04241846664645dc1675f01f2d7258650257e7d96b3e587b2b5bd9002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, RELEASE=main-290212a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=) 2024-02-10T08:14:36.770 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:36 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-10T08:14:36.770 INFO:journalctl@ceph.mon.a.smithi070.stdout:Feb 10 08:14:36 smithi070 systemd[1]: Stopped Ceph mon.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T08:14:36.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.a.service' 2024-02-10T08:14:36.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:14:36.812 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-10T08:14:36.812 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-10T08:14:36.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.c 2024-02-10T08:14:37.148 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:36 smithi070 systemd[1]: Stopping Ceph mon.c for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-c[39917]: 2024-02-10T08:14:37.146+0000 7f93827fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-c[39917]: 2024-02-10T08:14:37.146+0000 7f93827fa700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:37 smithi070 podman[155737]: 2024-02-10 08:14:37.19831783 +0000 UTC m=+0.168685584 container died 619b6dfbe927f4505c2f03259bef16579a5436e17291de230c2afc0beb1bcc59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-c, org.label-schema.build-date=20240131, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:37] ENGINE Bus STOPPING 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:37] ENGINE Bus STOPPED 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:37] ENGINE Bus STARTING 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:37] ENGINE Serving on http://:::9283 2024-02-10T08:14:37.474 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:37 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y[29785]: [10/Feb/2024:08:14:37] ENGINE Bus STARTED 2024-02-10T08:14:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:37 smithi070 podman[155737]: 2024-02-10 08:14:37.564431563 +0000 UTC m=+0.534799316 container cleanup 619b6dfbe927f4505c2f03259bef16579a5436e17291de230c2afc0beb1bcc59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-c, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2) 2024-02-10T08:14:37.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Feb 10 08:14:37 smithi070 bash[155737]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-c 2024-02-10T08:14:37.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.c.service' 2024-02-10T08:14:38.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:14:38.009 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-10T08:14:38.009 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-10T08:14:38.010 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.b 2024-02-10T08:14:38.505 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:38 smithi155 systemd[1]: Stopping Ceph mon.b for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:39.005 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-b[32682]: 2024-02-10T08:14:38.553+0000 7f949d588700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:14:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-b[32682]: 2024-02-10T08:14:38.553+0000 7f949d588700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-10T08:14:39.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:38 smithi155 podman[75263]: 2024-02-10 08:14:38.759230888 +0000 UTC m=+0.536687640 container died 57da29113b890e5452fd21f1625ba3b3a40607b0c4c632d985393808fe4942a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-10T08:14:39.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:39 smithi155 podman[75263]: 2024-02-10 08:14:39.196462931 +0000 UTC m=+0.973919676 container cleanup 57da29113b890e5452fd21f1625ba3b3a40607b0c4c632d985393808fe4942a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-b, RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-10T08:14:39.390 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 10 08:14:39 smithi155 bash[75263]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mon-b 2024-02-10T08:14:39.598 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mon.b.service' 2024-02-10T08:14:39.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:14:39.679 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-10T08:14:39.680 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-10T08:14:39.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.y 2024-02-10T08:14:39.987 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:39 smithi070 systemd[1]: Stopping Ceph mgr.y for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:40.242 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:39 smithi070 podman[155867]: 2024-02-10 08:14:39.991463429 +0000 UTC m=+0.120972612 container died 2a1360cd8c565f58695d77a7743ce6f36c949b65d577b24eaa2b66c2c0f9908f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-290212a, CEPH_POINT_RELEASE=) 2024-02-10T08:14:40.620 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:40 smithi070 podman[155867]: 2024-02-10 08:14:40.393757738 +0000 UTC m=+0.523266921 container cleanup 2a1360cd8c565f58695d77a7743ce6f36c949b65d577b24eaa2b66c2c0f9908f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y, RELEASE=main-290212a, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2) 2024-02-10T08:14:40.621 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Feb 10 08:14:40 smithi070 bash[155867]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-y 2024-02-10T08:14:40.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.y.service' 2024-02-10T08:14:40.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:14:40.798 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-10T08:14:40.799 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-10T08:14:40.799 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.x 2024-02-10T08:14:41.103 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 08:14:40 smithi155 systemd[1]: Stopping Ceph mgr.x for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:41.394 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 08:14:41 smithi155 podman[75390]: 2024-02-10 08:14:41.115934927 +0000 UTC m=+0.118903664 container died 1a7848ffa62202410a65257fdb9ed659b64ae3083418323f3772a751e21c568d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x, RELEASE=main-290212a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux ) 2024-02-10T08:14:41.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 08:14:41 smithi155 podman[75390]: 2024-02-10 08:14:41.47688953 +0000 UTC m=+0.479858267 container cleanup 1a7848ffa62202410a65257fdb9ed659b64ae3083418323f3772a751e21c568d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, CEPH_POINT_RELEASE=) 2024-02-10T08:14:41.755 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 10 08:14:41 smithi155 bash[75390]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-mgr-x 2024-02-10T08:14:41.894 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@mgr.x.service' 2024-02-10T08:14:41.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:14:41.934 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-10T08:14:41.934 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-10T08:14:41.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.0 2024-02-10T08:14:42.241 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:41 smithi070 systemd[1]: Stopping Ceph osd.0 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:42.741 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0[43929]: 2024-02-10T08:14:42.447+0000 7f59afa25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:14:42.742 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0[43929]: 2024-02-10T08:14:42.447+0000 7f59afa25700 -1 osd.0 839 *** Got signal Terminated *** 2024-02-10T08:14:42.742 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:42 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0[43929]: 2024-02-10T08:14:42.447+0000 7f59afa25700 -1 osd.0 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:14:48.164 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:47 smithi070 podman[155987]: 2024-02-10 08:14:47.854773975 +0000 UTC m=+5.733982144 container died 6f724c7e761068dd72a77b0ecfeae53d59a39cd0d944f32d5c45ba6ac868af60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-10T08:14:48.491 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:48 smithi070 podman[155987]: 2024-02-10 08:14:48.164027316 +0000 UTC m=+6.043235484 container cleanup 6f724c7e761068dd72a77b0ecfeae53d59a39cd0d944f32d5c45ba6ac868af60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.license=GPLv2) 2024-02-10T08:14:48.491 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:48 smithi070 bash[155987]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0 2024-02-10T08:14:48.492 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:48 smithi070 podman[156004]: 2024-02-10 08:14:48.398794841 +0000 UTC m=+0.540953957 container remove 6f724c7e761068dd72a77b0ecfeae53d59a39cd0d944f32d5c45ba6ac868af60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-10T08:14:48.992 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:48 smithi070 podman[156095]: 2024-02-10 08:14:48.62454523 +0000 UTC m=+0.017732665 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:14:48.992 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:48 smithi070 podman[156095]: 2024-02-10 08:14:48.732957087 +0000 UTC m=+0.126144522 container create a121d69c2e88371abf7d7a6518d599c68baf25c95c9dedb97678228ba1787fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, RELEASE=main-290212a, ceph=True, GIT_BRANCH=HEAD) 2024-02-10T08:14:48.992 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:48 smithi070 podman[156095]: 2024-02-10 08:14:48.943672005 +0000 UTC m=+0.336859440 container init a121d69c2e88371abf7d7a6518d599c68baf25c95c9dedb97678228ba1787fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, io.buildah.version=1.33.2) 2024-02-10T08:14:48.992 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:48 smithi070 podman[156095]: 2024-02-10 08:14:48.946826971 +0000 UTC m=+0.340014413 container start a121d69c2e88371abf7d7a6518d599c68baf25c95c9dedb97678228ba1787fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-10T08:14:49.491 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:49 smithi070 podman[156095]: 2024-02-10 08:14:49.026989669 +0000 UTC m=+0.420177121 container attach a121d69c2e88371abf7d7a6518d599c68baf25c95c9dedb97678228ba1787fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-10T08:14:49.492 INFO:journalctl@ceph.osd.0.smithi070.stdout:Feb 10 08:14:49 smithi070 podman[156095]: 2024-02-10 08:14:49.240681428 +0000 UTC m=+0.633868866 container died a121d69c2e88371abf7d7a6518d599c68baf25c95c9dedb97678228ba1787fd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-10T08:14:49.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.0.service' 2024-02-10T08:14:49.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:14:49.855 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-10T08:14:49.855 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-10T08:14:49.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.1 2024-02-10T08:14:50.050 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:49 smithi070 systemd[1]: Stopping Ceph osd.1 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:50.453 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:50 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1[49779]: 2024-02-10T08:14:50.200+0000 7fa89bd40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:14:50.505 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:50 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1[49779]: 2024-02-10T08:14:50.200+0000 7fa89bd40700 -1 osd.1 839 *** Got signal Terminated *** 2024-02-10T08:14:50.505 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:50 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1[49779]: 2024-02-10T08:14:50.200+0000 7fa89bd40700 -1 osd.1 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:14:55.920 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:55 smithi070 podman[156204]: 2024-02-10 08:14:55.637106839 +0000 UTC m=+5.544192771 container died da8eb37187d1b54367348f2ecffb3b3f85d22b80276428831ef0852d059b2684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-02-10T08:14:56.241 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 podman[156204]: 2024-02-10 08:14:56.041792308 +0000 UTC m=+5.948878234 container cleanup da8eb37187d1b54367348f2ecffb3b3f85d22b80276428831ef0852d059b2684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-10T08:14:56.242 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 bash[156204]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1 2024-02-10T08:14:56.576 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 podman[156232]: 2024-02-10 08:14:56.242875625 +0000 UTC m=+0.605791844 container remove da8eb37187d1b54367348f2ecffb3b3f85d22b80276428831ef0852d059b2684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-10T08:14:56.871 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 podman[156323]: 2024-02-10 08:14:56.479348365 +0000 UTC m=+0.017648039 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:14:56.872 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 podman[156323]: 2024-02-10 08:14:56.644094941 +0000 UTC m=+0.182394604 container create 37d78479717514379f45bd3e6fe45470c548bc8a8b5e6e6ab2660c438e767dd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-290212a, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-02-10T08:14:56.872 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 podman[156323]: 2024-02-10 08:14:56.871296089 +0000 UTC m=+0.409595757 container init 37d78479717514379f45bd3e6fe45470c548bc8a8b5e6e6ab2660c438e767dd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=main-290212a, ceph=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:14:57.159 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 podman[156323]: 2024-02-10 08:14:56.874737827 +0000 UTC m=+0.413037492 container start 37d78479717514379f45bd3e6fe45470c548bc8a8b5e6e6ab2660c438e767dd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-10T08:14:57.159 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:56 smithi070 podman[156323]: 2024-02-10 08:14:56.971192102 +0000 UTC m=+0.509491769 container attach 37d78479717514379f45bd3e6fe45470c548bc8a8b5e6e6ab2660c438e767dd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2) 2024-02-10T08:14:57.448 INFO:journalctl@ceph.osd.1.smithi070.stdout:Feb 10 08:14:57 smithi070 podman[156323]: 2024-02-10 08:14:57.159506029 +0000 UTC m=+0.697805688 container died 37d78479717514379f45bd3e6fe45470c548bc8a8b5e6e6ab2660c438e767dd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:14:57.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.1.service' 2024-02-10T08:14:57.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:14:57.797 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-10T08:14:57.797 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-10T08:14:57.798 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.2 2024-02-10T08:14:57.991 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:14:57 smithi070 systemd[1]: Stopping Ceph osd.2 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:14:58.491 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:14:58 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2[55564]: 2024-02-10T08:14:58.118+0000 7fcc1012c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:14:58.491 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:14:58 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2[55564]: 2024-02-10T08:14:58.118+0000 7fcc1012c700 -1 osd.2 839 *** Got signal Terminated *** 2024-02-10T08:14:58.492 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:14:58 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2[55564]: 2024-02-10T08:14:58.118+0000 7fcc1012c700 -1 osd.2 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:15:03.828 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:03 smithi070 podman[156427]: 2024-02-10 08:15:03.571732399 +0000 UTC m=+5.547232724 container died df60bef4fed0dcdda36ac9e8b63be82f93b6f4168f1169f251c5d2619fa5b249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-10T08:15:04.111 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:03 smithi070 podman[156427]: 2024-02-10 08:15:03.918121139 +0000 UTC m=+5.893621468 container cleanup df60bef4fed0dcdda36ac9e8b63be82f93b6f4168f1169f251c5d2619fa5b249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:15:04.111 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:03 smithi070 bash[156427]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2 2024-02-10T08:15:04.491 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:04 smithi070 podman[156444]: 2024-02-10 08:15:04.110847635 +0000 UTC m=+0.534751561 container remove df60bef4fed0dcdda36ac9e8b63be82f93b6f4168f1169f251c5d2619fa5b249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-10T08:15:04.839 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:04 smithi070 podman[156534]: 2024-02-10 08:15:04.437899612 +0000 UTC m=+0.018161481 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:15:04.839 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:04 smithi070 podman[156534]: 2024-02-10 08:15:04.553353208 +0000 UTC m=+0.133615076 container create b80c3ffa460dea72fdd391296a2e175eaae20331f2e933bfbd206d6766656644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-10T08:15:04.839 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:04 smithi070 podman[156534]: 2024-02-10 08:15:04.755591662 +0000 UTC m=+0.335853533 container init b80c3ffa460dea72fdd391296a2e175eaae20331f2e933bfbd206d6766656644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_CLEAN=True) 2024-02-10T08:15:04.839 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:04 smithi070 podman[156534]: 2024-02-10 08:15:04.759755005 +0000 UTC m=+0.340016875 container start b80c3ffa460dea72fdd391296a2e175eaae20331f2e933bfbd206d6766656644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T08:15:04.839 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:04 smithi070 podman[156534]: 2024-02-10 08:15:04.83880952 +0000 UTC m=+0.419071397 container attach b80c3ffa460dea72fdd391296a2e175eaae20331f2e933bfbd206d6766656644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True) 2024-02-10T08:15:05.241 INFO:journalctl@ceph.osd.2.smithi070.stdout:Feb 10 08:15:05 smithi070 podman[156534]: 2024-02-10 08:15:05.050215203 +0000 UTC m=+0.630477080 container died b80c3ffa460dea72fdd391296a2e175eaae20331f2e933bfbd206d6766656644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:15:05.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.2.service' 2024-02-10T08:15:05.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:05.656 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-10T08:15:05.656 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-10T08:15:05.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.3 2024-02-10T08:15:05.979 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:05 smithi070 systemd[1]: Stopping Ceph osd.3 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:06.241 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3[61781]: 2024-02-10T08:15:05.978+0000 7f2aeb96b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:15:06.241 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3[61781]: 2024-02-10T08:15:05.978+0000 7f2aeb96b700 -1 osd.3 839 *** Got signal Terminated *** 2024-02-10T08:15:06.242 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:05 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3[61781]: 2024-02-10T08:15:05.978+0000 7f2aeb96b700 -1 osd.3 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:15:09.506 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:09 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:09.111+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:10.407 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:10 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:10.123+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:11.506 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:11.131+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:11.099Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-02-10T08:15:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:11.099Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-02-10T08:15:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:11.101Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-02-10T08:15:11.507 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:11 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:11.101Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-02-10T08:15:11.721 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:11 smithi070 podman[156640]: 2024-02-10 08:15:11.459720187 +0000 UTC m=+5.578606231 container died f798d7b75dd826a5eac11617dd524a3ef5dc24e02a468592bac32ed9eae7489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.build-date=20240131, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-10T08:15:11.991 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:11 smithi070 podman[156640]: 2024-02-10 08:15:11.819443284 +0000 UTC m=+5.938329337 container cleanup f798d7b75dd826a5eac11617dd524a3ef5dc24e02a468592bac32ed9eae7489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0) 2024-02-10T08:15:11.991 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:11 smithi070 bash[156640]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3 2024-02-10T08:15:12.445 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:12 smithi070 podman[156659]: 2024-02-10 08:15:12.10390042 +0000 UTC m=+0.640242656 container remove f798d7b75dd826a5eac11617dd524a3ef5dc24e02a468592bac32ed9eae7489a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T08:15:12.505 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:12 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:12.093+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:12.724 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:12 smithi070 podman[156751]: 2024-02-10 08:15:12.347811303 +0000 UTC m=+0.018016425 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:15:12.724 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:12 smithi070 podman[156751]: 2024-02-10 08:15:12.446361009 +0000 UTC m=+0.116566161 container create 9e4f821c5ebf461556ee350b0b802db36d244d49c67562ac2a79d17113e19c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-10T08:15:12.724 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:12 smithi070 podman[156751]: 2024-02-10 08:15:12.648700012 +0000 UTC m=+0.318905133 container init 9e4f821c5ebf461556ee350b0b802db36d244d49c67562ac2a79d17113e19c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-10T08:15:12.724 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:12 smithi070 podman[156751]: 2024-02-10 08:15:12.651785275 +0000 UTC m=+0.321990386 container start 9e4f821c5ebf461556ee350b0b802db36d244d49c67562ac2a79d17113e19c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2) 2024-02-10T08:15:12.724 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:12 smithi070 podman[156751]: 2024-02-10 08:15:12.723634821 +0000 UTC m=+0.393839928 container attach 9e4f821c5ebf461556ee350b0b802db36d244d49c67562ac2a79d17113e19c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.build-date=20240131) 2024-02-10T08:15:12.992 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:12 smithi070 podman[156751]: 2024-02-10 08:15:12.937871781 +0000 UTC m=+0.608076895 container died 9e4f821c5ebf461556ee350b0b802db36d244d49c67562ac2a79d17113e19c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-02-10T08:15:13.005 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:12 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:12.741+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:13.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.3.service' 2024-02-10T08:15:13.491 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:13 smithi070 podman[156777]: 2024-02-10 08:15:13.440433292 +0000 UTC m=+0.497277941 container remove 9e4f821c5ebf461556ee350b0b802db36d244d49c67562ac2a79d17113e19c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-10T08:15:13.491 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:13 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-10T08:15:13.491 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:13 smithi070 systemd[1]: Stopped Ceph osd.3 for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T08:15:13.492 INFO:journalctl@ceph.osd.3.smithi070.stdout:Feb 10 08:15:13 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.3.service: Consumed 57.675s CPU time. 2024-02-10T08:15:13.505 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:13 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:13.096+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:13.505 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:13 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:13.124+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:13.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:13 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:13.266+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:13.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:13.509 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-10T08:15:13.509 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-10T08:15:13.509 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.4 2024-02-10T08:15:13.829 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:13 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:13.776+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:13.829 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:13 smithi155 systemd[1]: Stopping Ceph osd.4 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:13.829 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:13 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:13.827+0000 7f4981f57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:15:13.829 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:13 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:13.827+0000 7f4981f57700 -1 osd.4 839 *** Got signal Terminated *** 2024-02-10T08:15:13.830 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:13 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:13.827+0000 7f4981f57700 -1 osd.4 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:15:14.506 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:14 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:14.116+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:14.506 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:14 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:14.097+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:14.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:14 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:14.230+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:15.005 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:14 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:14.739+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:15.507 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:15 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:15.071+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:15.511 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:15 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:15.159+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:15.511 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:15 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:15.263+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:16.006 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:15 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:15.751+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:16.505 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:16 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:16.199+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:16.506 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:16 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:16.083+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:16.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:16 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:16.273+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:17.052 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:16 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:16.779+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:17.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:17 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:17.277+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:17.506 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:17 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:17.230+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:17.506 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:17 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:17.051+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:18.252 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:18.019+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:18.253 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:17 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:17.821+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:18.253 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:17 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:17.821+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:18.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:18.288+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:18.506 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4[37672]: 2024-02-10T08:15:18.251+0000 7f497e54b700 -1 osd.4 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:43.698625+0000 front 2024-02-10T08:14:43.698565+0000 (oldest deadline 2024-02-10T08:15:08.398525+0000) 2024-02-10T08:15:19.191 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:19.011+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:19.191 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:19.011+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:19.191 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:18.858+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:19.192 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:18 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:18.858+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:19.480 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:19 smithi155 podman[75540]: 2024-02-10 08:15:19.191076832 +0000 UTC m=+5.483294447 container died 2b031a06934224c73fd2a53898652567be252a82002ffc6161c8135cc8dc4ee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131) 2024-02-10T08:15:19.481 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:19.291+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:19.481 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:19.291+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:19.755 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:19 smithi155 podman[75540]: 2024-02-10 08:15:19.570416363 +0000 UTC m=+5.862633980 container cleanup 2b031a06934224c73fd2a53898652567be252a82002ffc6161c8135cc8dc4ee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T08:15:19.755 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:19 smithi155 bash[75540]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4 2024-02-10T08:15:20.142 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:19 smithi155 podman[75557]: 2024-02-10 08:15:19.830253056 +0000 UTC m=+0.630048208 container remove 2b031a06934224c73fd2a53898652567be252a82002ffc6161c8135cc8dc4ee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-10T08:15:20.142 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:19.894+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:20.142 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:19.894+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:20.142 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:19.996+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:20.142 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:19 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:19.996+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:20.467 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:20.263+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:20.467 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:20.263+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:20.467 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:20 smithi155 podman[75647]: 2024-02-10 08:15:20.044268013 +0000 UTC m=+0.018573996 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:15:20.467 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:20 smithi155 podman[75647]: 2024-02-10 08:15:20.147652674 +0000 UTC m=+0.121958655 container create d822cf728187c53f411e22dcebbfd8a99f403fb38cb5b7bcdb7f379f51566baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-10T08:15:20.467 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:20 smithi155 podman[75647]: 2024-02-10 08:15:20.375108234 +0000 UTC m=+0.349414205 container init d822cf728187c53f411e22dcebbfd8a99f403fb38cb5b7bcdb7f379f51566baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:15:20.467 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:20 smithi155 podman[75647]: 2024-02-10 08:15:20.378065035 +0000 UTC m=+0.352371006 container start d822cf728187c53f411e22dcebbfd8a99f403fb38cb5b7bcdb7f379f51566baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T08:15:20.755 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:20 smithi155 podman[75647]: 2024-02-10 08:15:20.466861248 +0000 UTC m=+0.441167233 container attach d822cf728187c53f411e22dcebbfd8a99f403fb38cb5b7bcdb7f379f51566baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-02-10T08:15:20.756 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 10 08:15:20 smithi155 podman[75647]: 2024-02-10 08:15:20.669435812 +0000 UTC m=+0.643741775 container died d822cf728187c53f411e22dcebbfd8a99f403fb38cb5b7bcdb7f379f51566baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-4-deactivate, RELEASE=main-290212a, org.label-schema.build-date=20240131, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T08:15:21.130 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:20.995+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:21.130 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:20.995+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:21.130 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:20.870+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:21.130 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:20 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:20.870+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:21.283 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.4.service' 2024-02-10T08:15:21.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:21.325 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-10T08:15:21.325 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-10T08:15:21.326 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.5 2024-02-10T08:15:21.389 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:21.234+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:21.389 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:21.234+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:21.649 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:21 smithi155 systemd[1]: Stopping Ceph osd.5 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:21.915 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:21.648+0000 7fb13a7de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:15:21.915 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:21.648+0000 7fb13a7de700 -1 osd.5 839 *** Got signal Terminated *** 2024-02-10T08:15:21.915 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:21.648+0000 7fb13a7de700 -1 osd.5 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:15:21.915 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:21.888+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:21.915 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:21.888+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:22.255 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:21.990+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:22.256 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:21 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:21.990+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:22.755 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:22 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:22.284+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:22.756 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:22 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:22.284+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:23.255 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:22 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:22.879+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:23.256 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:22 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:22.879+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:23.256 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:22 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:22.970+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:23.256 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:22 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:22.970+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:23.756 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:23 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:23.260+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:23.756 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:23 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:23.260+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:24.232 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:23 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:23.888+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:24.232 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:23 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:23.888+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:24.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:23 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:23.920+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:24.233 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:23 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:23.920+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:24.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:24 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:24.231+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:24.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:24 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:24.231+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:25.229 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:24 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:24.846+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:25.229 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:24 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:24.846+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:25.229 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:24 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:24.846+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:14:59.192666+0000 front 2024-02-10T08:14:59.192858+0000 (oldest deadline 2024-02-10T08:15:23.892774+0000) 2024-02-10T08:15:25.230 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:24 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:24.874+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:25.230 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:24 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:24.874+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:25.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:25.228+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:25.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:25.228+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:26.252 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:25.847+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:42.989658+0000 front 2024-02-10T08:14:42.989366+0000 (oldest deadline 2024-02-10T08:15:11.790075+0000) 2024-02-10T08:15:26.252 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:25.847+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.791688+0000 front 2024-02-10T08:14:52.791676+0000 (oldest deadline 2024-02-10T08:15:17.491584+0000) 2024-02-10T08:15:26.252 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5[43587]: 2024-02-10T08:15:25.847+0000 7fb132dca700 -1 osd.5 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:14:59.192666+0000 front 2024-02-10T08:14:59.192858+0000 (oldest deadline 2024-02-10T08:15:23.892774+0000) 2024-02-10T08:15:26.252 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:25.909+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:26.252 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:25.909+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:26.252 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:25 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:25.909+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:26.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:26.250+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:26.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:26.250+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:26.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:26.250+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:27.228 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:27 smithi155 podman[75756]: 2024-02-10 08:15:27.148555824 +0000 UTC m=+5.604487943 container died 736352943ca58d26744b1485009ad75ebfbe8f04f22f33807b3960d237ac00c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, RELEASE=main-290212a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.2) 2024-02-10T08:15:27.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:26.899+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:27.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:26.899+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:27.229 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:26 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:26.899+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:27.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:27.227+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:27.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:27.227+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:27.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:27.227+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:27.783 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:27 smithi155 podman[75756]: 2024-02-10 08:15:27.506967191 +0000 UTC m=+5.962899320 container cleanup 736352943ca58d26744b1485009ad75ebfbe8f04f22f33807b3960d237ac00c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-10T08:15:27.783 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:27 smithi155 bash[75756]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5 2024-02-10T08:15:28.109 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:27 smithi155 podman[75772]: 2024-02-10 08:15:27.783313754 +0000 UTC m=+0.629833246 container remove 736352943ca58d26744b1485009ad75ebfbe8f04f22f33807b3960d237ac00c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, RELEASE=main-290212a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-10T08:15:28.109 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:27.871+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:28.109 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:27.871+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:28.110 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:27 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:27.871+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:28.370 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:28.274+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:28.370 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:28.274+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:28.371 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:28.274+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:28.371 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:28 smithi155 podman[75868]: 2024-02-10 08:15:28.109353855 +0000 UTC m=+0.095595353 container create f5f8c1f2f802d575213da36251a5cba1445c69d8da2cd1a30e9bf777dc2e49bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T08:15:28.371 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:28 smithi155 podman[75868]: 2024-02-10 08:15:28.031626684 +0000 UTC m=+0.017868194 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:15:28.371 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:28 smithi155 podman[75868]: 2024-02-10 08:15:28.296005631 +0000 UTC m=+0.282247136 container init f5f8c1f2f802d575213da36251a5cba1445c69d8da2cd1a30e9bf777dc2e49bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:15:28.371 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:28 smithi155 podman[75868]: 2024-02-10 08:15:28.299612128 +0000 UTC m=+0.285853626 container start f5f8c1f2f802d575213da36251a5cba1445c69d8da2cd1a30e9bf777dc2e49bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2) 2024-02-10T08:15:28.371 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:28 smithi155 podman[75868]: 2024-02-10 08:15:28.370229807 +0000 UTC m=+0.356471306 container attach f5f8c1f2f802d575213da36251a5cba1445c69d8da2cd1a30e9bf777dc2e49bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2) 2024-02-10T08:15:28.755 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 10 08:15:28 smithi155 podman[75868]: 2024-02-10 08:15:28.587325954 +0000 UTC m=+0.573567452 container died f5f8c1f2f802d575213da36251a5cba1445c69d8da2cd1a30e9bf777dc2e49bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:15:29.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:28.893+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:29.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:28.893+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:29.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:28 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:28.893+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:29.322 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.5.service' 2024-02-10T08:15:29.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:29.404 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-10T08:15:29.404 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-10T08:15:29.404 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.6 2024-02-10T08:15:29.488 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:29.226+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:29.488 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:29.226+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:29.488 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:29.226+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:29.755 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:29 smithi155 systemd[1]: Stopping Ceph osd.6 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:29.756 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:29.726+0000 7fac58cfc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:15:29.756 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:29.726+0000 7fac58cfc700 -1 osd.6 839 *** Got signal Terminated *** 2024-02-10T08:15:29.756 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:29.726+0000 7fac58cfc700 -1 osd.6 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:15:30.237 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:29.897+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:30.238 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:29.897+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:30.238 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:29 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:29.897+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:30.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:30.235+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:30.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:30.235+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:30.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:30.235+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:31.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:30.923+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:31.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:30.923+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:31.228 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:30 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:30.923+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:31.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:31.227+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:31.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:31.227+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:31.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:31.227+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:32.196 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:31.895+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:32.196 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:31.895+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:32.196 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:31 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:31.895+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:32.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:32.195+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:32.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:32.195+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:32.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:32.195+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:32.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:32.195+0000 7fb21fd6a700 -1 osd.7 839 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24469.0:2207 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e839) 2024-02-10T08:15:33.222 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:32.922+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:33.222 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:32.922+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:33.222 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:32.922+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:33.223 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:32 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:32.922+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:09.738879+0000 front 2024-02-10T08:15:09.738897+0000 (oldest deadline 2024-02-10T08:15:32.638804+0000) 2024-02-10T08:15:33.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:33.220+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:33.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:33.220+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:33.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:33.220+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:33.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:33.220+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:33.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:33.220+0000 7fb21fd6a700 -1 osd.7 839 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24469.0:2207 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e839) 2024-02-10T08:15:34.225 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:33.964+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:46.936333+0000 front 2024-02-10T08:14:46.936301+0000 (oldest deadline 2024-02-10T08:15:12.836072+0000) 2024-02-10T08:15:34.225 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:33.964+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.836644+0000 front 2024-02-10T08:14:52.836631+0000 (oldest deadline 2024-02-10T08:15:18.136615+0000) 2024-02-10T08:15:34.225 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:33.964+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.737966+0000 front 2024-02-10T08:15:02.738053+0000 (oldest deadline 2024-02-10T08:15:25.637899+0000) 2024-02-10T08:15:34.225 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:33 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6[49173]: 2024-02-10T08:15:33.964+0000 7fac512e8700 -1 osd.6 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:09.738879+0000 front 2024-02-10T08:15:09.738897+0000 (oldest deadline 2024-02-10T08:15:32.638804+0000) 2024-02-10T08:15:34.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:34.224+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:34.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:34.224+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:34.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:34.224+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:34.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:34 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:34.224+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:35.503 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:35 smithi155 podman[75972]: 2024-02-10 08:15:35.260838498 +0000 UTC m=+5.633925492 container died ad1e9a491dfb96e6c79c311156146dda2264cccaa0f522da64e2d58cd955ae21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-10T08:15:35.504 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:35 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:35.235+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:35.504 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:35 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:35.235+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:35.504 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:35 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:35.235+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:35.504 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:35 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:35.235+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:35.755 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:35 smithi155 podman[75972]: 2024-02-10 08:15:35.602139604 +0000 UTC m=+5.975226597 container cleanup ad1e9a491dfb96e6c79c311156146dda2264cccaa0f522da64e2d58cd955ae21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, org.label-schema.build-date=20240131) 2024-02-10T08:15:35.756 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:35 smithi155 bash[75972]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6 2024-02-10T08:15:36.191 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:35 smithi155 podman[75989]: 2024-02-10 08:15:35.786793897 +0000 UTC m=+0.524822656 container remove ad1e9a491dfb96e6c79c311156146dda2264cccaa0f522da64e2d58cd955ae21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131) 2024-02-10T08:15:36.192 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:36 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:36.190+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:36.192 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:36 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:36.190+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:36.192 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:36 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:36.190+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:36.498 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:36 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:36.190+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:36.499 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:36 smithi155 podman[76082]: 2024-02-10 08:15:36.125089309 +0000 UTC m=+0.017771252 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:15:36.499 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:36 smithi155 podman[76082]: 2024-02-10 08:15:36.254400471 +0000 UTC m=+0.147082396 container create bf6b757f3953dc39385f4205e51acca3062386028d99f3fa2a81f436215b7d37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6-deactivate, RELEASE=main-290212a, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T08:15:36.499 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:36 smithi155 podman[76082]: 2024-02-10 08:15:36.498575717 +0000 UTC m=+0.391257656 container init bf6b757f3953dc39385f4205e51acca3062386028d99f3fa2a81f436215b7d37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T08:15:36.756 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:36 smithi155 podman[76082]: 2024-02-10 08:15:36.502209166 +0000 UTC m=+0.394891092 container start bf6b757f3953dc39385f4205e51acca3062386028d99f3fa2a81f436215b7d37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T08:15:36.756 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:36 smithi155 podman[76082]: 2024-02-10 08:15:36.598494977 +0000 UTC m=+0.491176903 container attach bf6b757f3953dc39385f4205e51acca3062386028d99f3fa2a81f436215b7d37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0) 2024-02-10T08:15:37.076 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:36 smithi155 podman[76082]: 2024-02-10 08:15:36.789264916 +0000 UTC m=+0.681946843 container died bf6b757f3953dc39385f4205e51acca3062386028d99f3fa2a81f436215b7d37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=main-290212a) 2024-02-10T08:15:37.327 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 10 08:15:37 smithi155 podman[76107]: 2024-02-10 08:15:37.310530443 +0000 UTC m=+0.516497949 container remove bf6b757f3953dc39385f4205e51acca3062386028d99f3fa2a81f436215b7d37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-10T08:15:37.327 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:37.150+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:37.327 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:37.150+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:37.327 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:37.150+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:37.327 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:37.150+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:37.332 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.6.service' 2024-02-10T08:15:37.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:37.373 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-10T08:15:37.373 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-10T08:15:37.373 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.7 2024-02-10T08:15:37.689 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 systemd[1]: Stopping Ceph osd.7 for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:38.006 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:37.688+0000 7fb223776700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:15:38.006 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:37.688+0000 7fb223776700 -1 osd.7 839 *** Got signal Terminated *** 2024-02-10T08:15:38.006 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:37 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:37.688+0000 7fb223776700 -1 osd.7 839 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:15:38.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:38.134+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:38.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:38.134+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:38.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:38.134+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:38.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:38 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:38.134+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:39.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:39.171+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:39.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:39.171+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:39.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:39.171+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:39.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:39 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:39.171+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:40.407 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:40.135+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:40.407 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:40.135+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:40.407 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:40.135+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:40.407 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:40.135+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:40.407 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:40 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:40.135+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.155:6802 osd.4 since back 2024-02-10T08:15:18.315506+0000 front 2024-02-10T08:15:18.315606+0000 (oldest deadline 2024-02-10T08:15:39.415695+0000) 2024-02-10T08:15:41.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:41.153+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:41.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:41.153+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:41.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:41.153+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:41.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:41.153+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:41.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:41 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:41.153+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.155:6802 osd.4 since back 2024-02-10T08:15:18.315506+0000 front 2024-02-10T08:15:18.315606+0000 (oldest deadline 2024-02-10T08:15:39.415695+0000) 2024-02-10T08:15:42.505 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:42 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:42.144+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6803 osd.0 since back 2024-02-10T08:14:47.312005+0000 front 2024-02-10T08:14:47.312042+0000 (oldest deadline 2024-02-10T08:15:12.611971+0000) 2024-02-10T08:15:42.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:42 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:42.144+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6807 osd.1 since back 2024-02-10T08:14:52.612431+0000 front 2024-02-10T08:14:52.612590+0000 (oldest deadline 2024-02-10T08:15:18.512443+0000) 2024-02-10T08:15:42.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:42 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:42.144+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6811 osd.2 since back 2024-02-10T08:15:02.613460+0000 front 2024-02-10T08:15:02.613367+0000 (oldest deadline 2024-02-10T08:15:25.513356+0000) 2024-02-10T08:15:42.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:42 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:42.144+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.70:6815 osd.3 since back 2024-02-10T08:15:08.314951+0000 front 2024-02-10T08:15:08.314967+0000 (oldest deadline 2024-02-10T08:15:33.014802+0000) 2024-02-10T08:15:42.506 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:42 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7[54864]: 2024-02-10T08:15:42.144+0000 7fb21fd6a700 -1 osd.7 839 heartbeat_check: no reply from 172.21.15.155:6802 osd.4 since back 2024-02-10T08:15:18.315506+0000 front 2024-02-10T08:15:18.315606+0000 (oldest deadline 2024-02-10T08:15:39.415695+0000) 2024-02-10T08:15:43.322 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:43 smithi155 podman[76186]: 2024-02-10 08:15:43.023544057 +0000 UTC m=+5.424124282 container died c003c1ebc8ac9f3d612555ace7e40c6ff6f65127e6ddd526da312b07a012a0eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T08:15:43.639 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:43 smithi155 podman[76186]: 2024-02-10 08:15:43.421348866 +0000 UTC m=+5.821929093 container cleanup c003c1ebc8ac9f3d612555ace7e40c6ff6f65127e6ddd526da312b07a012a0eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7, GIT_BRANCH=HEAD, RELEASE=main-290212a, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T08:15:43.639 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:43 smithi155 bash[76186]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7 2024-02-10T08:15:44.005 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:43 smithi155 podman[76215]: 2024-02-10 08:15:43.639086054 +0000 UTC m=+0.610878732 container remove c003c1ebc8ac9f3d612555ace7e40c6ff6f65127e6ddd526da312b07a012a0eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a) 2024-02-10T08:15:44.309 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:44 smithi155 podman[76304]: 2024-02-10 08:15:43.943613417 +0000 UTC m=+0.017911473 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T08:15:44.310 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:44 smithi155 podman[76304]: 2024-02-10 08:15:44.048251243 +0000 UTC m=+0.122549288 container create 090f903ce1c50a378f0395ef7181d8e004d30be947ce4c8556446686bc5b7bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, org.label-schema.build-date=20240131) 2024-02-10T08:15:44.310 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:44 smithi155 podman[76304]: 2024-02-10 08:15:44.242631084 +0000 UTC m=+0.316929141 container init 090f903ce1c50a378f0395ef7181d8e004d30be947ce4c8556446686bc5b7bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-10T08:15:44.310 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:44 smithi155 podman[76304]: 2024-02-10 08:15:44.245799505 +0000 UTC m=+0.320097549 container start 090f903ce1c50a378f0395ef7181d8e004d30be947ce4c8556446686bc5b7bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-10T08:15:44.756 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:44 smithi155 podman[76304]: 2024-02-10 08:15:44.309175076 +0000 UTC m=+0.383473125 container attach 090f903ce1c50a378f0395ef7181d8e004d30be947ce4c8556446686bc5b7bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7-deactivate, RELEASE=main-290212a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T08:15:44.756 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:44 smithi155 podman[76304]: 2024-02-10 08:15:44.538255349 +0000 UTC m=+0.612553409 container died 090f903ce1c50a378f0395ef7181d8e004d30be947ce4c8556446686bc5b7bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , RELEASE=main-290212a, ceph=True) 2024-02-10T08:15:45.037 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 10 08:15:45 smithi155 podman[76330]: 2024-02-10 08:15:45.020636014 +0000 UTC m=+0.476847650 container remove 090f903ce1c50a378f0395ef7181d8e004d30be947ce4c8556446686bc5b7bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-10T08:15:45.042 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@osd.7.service' 2024-02-10T08:15:45.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:45.083 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-10T08:15:45.083 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-10T08:15:45.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@rgw.foo.a 2024-02-10T08:15:45.382 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 08:15:45 smithi070 systemd[1]: Stopping Ceph rgw.foo.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:45.648 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 08:15:45 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a[66916]: 2024-02-10T08:15:45.381+0000 7fd725410700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T08:15:45.648 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 08:15:45 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a[66916]: 2024-02-10T08:15:45.381+0000 7fd733c81ac0 -1 shutting down 2024-02-10T08:15:55.739 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 08:15:55 smithi070 bash[156857]: time="2024-02-10T08:15:55Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-10T08:15:55.740 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 08:15:55 smithi070 podman[156857]: 2024-02-10 08:15:55.40023869 +0000 UTC m=+10.118607855 container died 9000c620813b7a71066a449ef7f67c6beec7e0546a0843965fa1becf3cda5cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-10T08:15:55.991 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 08:15:55 smithi070 podman[156857]: 2024-02-10 08:15:55.829848022 +0000 UTC m=+10.548217195 container cleanup 9000c620813b7a71066a449ef7f67c6beec7e0546a0843965fa1becf3cda5cb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-10T08:15:55.992 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Feb 10 08:15:55 smithi070 bash[156857]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-rgw-foo-a 2024-02-10T08:15:56.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@rgw.foo.a.service' 2024-02-10T08:15:56.293 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:56.293 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-10T08:15:56.293 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-10T08:15:56.293 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-70142e68-c7e6-11ee-95b8-87774f69a715@prometheus.a 2024-02-10T08:15:56.623 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 systemd[1]: Stopping Ceph prometheus.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.622Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.623Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.624Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T08:15:56.882 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.624Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-10T08:15:56.883 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a[64733]: ts=2024-02-10T08:15:56.624Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-10T08:15:56.883 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 podman[76409]: 2024-02-10 08:15:56.633208229 +0000 UTC m=+0.142532884 container died 699fc1bc5a90ba308c649c08c7f105c5a5c31ea651df46458f14aed63faff6e7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T08:15:57.255 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 podman[76409]: 2024-02-10 08:15:56.972512492 +0000 UTC m=+0.481837154 container cleanup 699fc1bc5a90ba308c649c08c7f105c5a5c31ea651df46458f14aed63faff6e7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T08:15:57.255 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:56 smithi155 bash[76409]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T08:15:57.256 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 10 08:15:57 smithi155 podman[76423]: 2024-02-10 08:15:57.123811504 +0000 UTC m=+0.491493709 container remove 699fc1bc5a90ba308c649c08c7f105c5a5c31ea651df46458f14aed63faff6e7 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T08:15:57.349 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-70142e68-c7e6-11ee-95b8-87774f69a715@prometheus.a.service' 2024-02-10T08:15:57.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:15:57.387 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-10T08:15:57.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T08:15:57.572 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T08:15:59.492 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:15:59 smithi070 systemd[1]: Stopping Ceph alertmanager.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:15:59.894 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:15:59 smithi070 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a[76485]: ts=2024-02-10T08:15:59.576Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-10T08:15:59.894 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:15:59 smithi070 podman[157360]: 2024-02-10 08:15:59.617567797 +0000 UTC m=+0.139394577 container died d1b6c45deea91b7e755d36a4138dd4289e0e678df501703dfdc7fa97c2d92c71 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T08:16:00.236 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:15:59 smithi070 podman[157360]: 2024-02-10 08:15:59.984988592 +0000 UTC m=+0.506815372 container cleanup d1b6c45deea91b7e755d36a4138dd4289e0e678df501703dfdc7fa97c2d92c71 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T08:16:00.236 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:15:59 smithi070 bash[157360]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T08:16:00.491 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:16:00 smithi070 podman[157374]: 2024-02-10 08:16:00.235932529 +0000 UTC m=+0.610131211 container remove d1b6c45deea91b7e755d36a4138dd4289e0e678df501703dfdc7fa97c2d92c71 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T08:16:00.492 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:16:00 smithi070 podman[157374]: 2024-02-10 08:16:00.311582189 +0000 UTC m=+0.685780826 volume remove 9f937efc2bb820bbe11b45c33ae7ad4272d2ac28ca2aa5e131d5a2fb122efb28 2024-02-10T08:16:00.492 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:16:00 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-10T08:16:00.492 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:16:00 smithi070 systemd[1]: Stopped Ceph alertmanager.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T08:16:00.492 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Feb 10 08:16:00 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@alertmanager.a.service: Consumed 2.401s CPU time. 2024-02-10T08:16:00.991 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:00 smithi070 systemd[1]: Stopping Ceph node-exporter.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:16:00.992 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:00 smithi070 podman[157487]: 2024-02-10 08:16:00.883405881 +0000 UTC m=+0.113343641 container died 1ae0911b3151117fe5a9eda85e9f2554fb3c44292969f82c849d7ef3c1edb11f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T08:16:01.492 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:01 smithi070 podman[157487]: 2024-02-10 08:16:01.230850918 +0000 UTC m=+0.460788677 container cleanup 1ae0911b3151117fe5a9eda85e9f2554fb3c44292969f82c849d7ef3c1edb11f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T08:16:01.492 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:01 smithi070 bash[157487]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a 2024-02-10T08:16:01.796 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:01 smithi070 podman[157501]: 2024-02-10 08:16:01.518697569 +0000 UTC m=+0.637681670 container remove 1ae0911b3151117fe5a9eda85e9f2554fb3c44292969f82c849d7ef3c1edb11f (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T08:16:01.796 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:01 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T08:16:01.796 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:01 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-10T08:16:01.797 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:01 smithi070 systemd[1]: Stopped Ceph node-exporter.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T08:16:01.797 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Feb 10 08:16:01 smithi070 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.a.service: Consumed 5.162s CPU time. 2024-02-10T08:16:02.266 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T08:16:02.457 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T08:16:03.978 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:03 smithi155 systemd[1]: Stopping Ceph iscsi.iscsi.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:16:04.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:03 smithi155 ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a[60013]: debug Shutdown received 2024-02-10T08:16:14.256 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:13 smithi155 bash[76821]: time="2024-02-10T08:16:13Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-10T08:16:14.256 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:13 smithi155 podman[76821]: 2024-02-10 08:16:13.994086687 +0000 UTC m=+10.109794698 container died 0217526817a5d1fe94969dcdbd172b285162dcedb425455864b730a5f5521177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.2, RELEASE=main-290212a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T08:16:14.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:14 smithi155 podman[76821]: 2024-02-10 08:16:14.400983618 +0000 UTC m=+10.516691629 container cleanup 0217526817a5d1fe94969dcdbd172b285162dcedb425455864b730a5f5521177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131) 2024-02-10T08:16:14.621 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:14 smithi155 bash[76821]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a 2024-02-10T08:16:14.901 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:14 smithi155 podman[76834]: 2024-02-10 08:16:14.627193306 +0000 UTC m=+0.633482956 container remove 0217526817a5d1fe94969dcdbd172b285162dcedb425455864b730a5f5521177 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T08:16:14.901 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:14 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-10T08:16:15.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:14 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-10T08:16:15.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:14 smithi155 systemd[1]: Stopped Ceph iscsi.iscsi.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T08:16:15.208 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 10 08:16:14 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Consumed 4.409s CPU time. 2024-02-10T08:16:16.505 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:16 smithi155 systemd[1]: Stopping Ceph grafana.a for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:16:16.951 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:16 smithi155 podman[77112]: 2024-02-10 08:16:16.620038536 +0000 UTC m=+0.107807263 container died 9da082eb524410417420f495ee35da41764fc3679748ec0b17c279708d1a5119 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, io.buildah.version=1.32.0, vcs-type=git, vendor=Red Hat, Inc., architecture=x86_64, release=239.1651231664, maintainer=Paul Cuzner , version=8.5, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container) 2024-02-10T08:16:16.951 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:16 smithi155 podman[77112]: 2024-02-10 08:16:16.951082119 +0000 UTC m=+0.438850843 container cleanup 9da082eb524410417420f495ee35da41764fc3679748ec0b17c279708d1a5119 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a, vendor=Red Hat, Inc., io.openshift.expose-services=, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, name=ubi8, vcs-type=git, distribution-scope=public, com.redhat.component=ubi8-container) 2024-02-10T08:16:17.255 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:16 smithi155 bash[77112]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a 2024-02-10T08:16:17.255 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:17 smithi155 podman[77126]: 2024-02-10 08:16:17.1272296 +0000 UTC m=+0.509938395 container remove 9da082eb524410417420f495ee35da41764fc3679748ec0b17c279708d1a5119 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-grafana-a, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, release=239.1651231664, vcs-type=git, io.buildah.version=1.32.0, maintainer=Paul Cuzner , vendor=Red Hat, Inc., distribution-scope=public, version=8.5, io.openshift.expose-services=) 2024-02-10T08:16:17.256 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:17 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T08:16:17.756 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:17 smithi155 systemd[1]: Stopping Ceph node-exporter.b for 70142e68-c7e6-11ee-95b8-87774f69a715... 2024-02-10T08:16:17.756 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:17 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-10T08:16:17.756 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:17 smithi155 systemd[1]: Stopped Ceph grafana.a for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T08:16:17.756 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 10 08:16:17 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@grafana.a.service: Consumed 18.092s CPU time. 2024-02-10T08:16:18.089 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:17 smithi155 podman[77240]: 2024-02-10 08:16:17.81567685 +0000 UTC m=+0.104253547 container died 9284088414a559955b951a8edefde7afd09c536315798d9224f92d9e2ebc566d (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T08:16:18.448 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:18 smithi155 podman[77240]: 2024-02-10 08:16:18.188675749 +0000 UTC m=+0.477252444 container cleanup 9284088414a559955b951a8edefde7afd09c536315798d9224f92d9e2ebc566d (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T08:16:18.448 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:18 smithi155 bash[77240]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b 2024-02-10T08:16:18.750 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:18 smithi155 podman[77254]: 2024-02-10 08:16:18.448042236 +0000 UTC m=+0.636787720 container remove 9284088414a559955b951a8edefde7afd09c536315798d9224f92d9e2ebc566d (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-70142e68-c7e6-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T08:16:18.750 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:18 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-10T08:16:18.750 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:18 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-10T08:16:18.750 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:18 smithi155 systemd[1]: Stopped Ceph node-exporter.b for 70142e68-c7e6-11ee-95b8-87774f69a715. 2024-02-10T08:16:18.751 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 10 08:16:18 smithi155 systemd[1]: ceph-70142e68-c7e6-11ee-95b8-87774f69a715@node-exporter.b.service: Consumed 5.087s CPU time. 2024-02-10T08:16:19.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:16:19.267 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:16:19.296 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-10T08:16:19.298 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217/remote/smithi070/crash 2024-02-10T08:16:19.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/crash -- . 2024-02-10T08:16:19.429 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T08:16:19.429 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2024-02-10T08:16:19.431 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217/remote/smithi155/crash 2024-02-10T08:16:19.432 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/crash -- . 2024-02-10T08:16:19.473 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T08:16:19.473 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-02-10T08:16:19.474 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-10T08:16:19.474 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T08:16:19.507 INFO:teuthology.orchestra.run.smithi070.stdout:2024-02-10T07:50:00.000131+0000 mon.a (mon.0) 2853 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-10T08:16:19.507 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-10T08:16:19.508 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[SEC\]' /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T08:16:19.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]' /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T08:16:19.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[WRN\]' /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-10T08:16:19.723 INFO:teuthology.orchestra.run.smithi070.stdout:2024-02-10T07:50:00.000131+0000 mon.a (mon.0) 2853 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-10T08:16:19.723 INFO:tasks.cephadm:Compressing logs... 2024-02-10T08:16:19.723 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T08:16:19.766 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T08:16:19.791 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T08:16:19.791 INFO:teuthology.orchestra.run.smithi070.stderr:: No such file or directory 2024-02-10T08:16:19.791 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.a.log 2024-02-10T08:16:19.793 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T08:16:19.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-10T08:16:19.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.a.log: /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.1.log 2024-02-10T08:16:19.793 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.3.log 2024-02-10T08:16:19.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log: gzip/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mgr.y.log 2024-02-10T08:16:19.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.0.log 2024-02-10T08:16:19.795 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T08:16:19.795 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-02-10T08:16:19.795 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.6.log 2024-02-10T08:16:19.795 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose --/var/log/ceph/cephadm.log: /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T08:16:19.795 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.4.log 2024-02-10T08:16:19.799 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.7.log 2024-02-10T08:16:19.800 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.5.log 2024-02-10T08:16:19.800 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mgr.x.log 2024-02-10T08:16:19.800 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/tcmu-runner.log 2024-02-10T08:16:19.800 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T08:16:19.800 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/tcmu-runner.log: 67.6% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/tcmu-runner.log.gz 2024-02-10T08:16:19.800 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log 2024-02-10T08:16:19.802 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mgr.y.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T08:16:19.802 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T08:16:19.803 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.cephadm.log: 81.6% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T08:16:19.803 INFO:teuthology.orchestra.run.smithi155.stderr: 85.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T08:16:19.803 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T08:16:19.803 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.b.log 2024-02-10T08:16:19.804 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-volume.log: /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log: 88.1% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T08:16:19.805 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T08:16:19.807 INFO:teuthology.orchestra.run.smithi155.stderr: 92.3% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mgr.x.log.gz 2024-02-10T08:16:19.811 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.0.log: 95.2% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T08:16:19.811 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log 2024-02-10T08:16:19.813 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.cephadm.log: 89.1% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T08:16:19.813 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.2.log 2024-02-10T08:16:19.813 INFO:teuthology.orchestra.run.smithi155.stderr: 90.8% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T08:16:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log: 93.4% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T08:16:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T08:16:19.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.c.log 2024-02-10T08:16:19.860 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-volume.log: /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.c.log: 93.5% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-10T08:16:19.865 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T08:16:20.343 INFO:teuthology.orchestra.run.smithi070.stderr: 90.2% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mgr.y.log.gz 2024-02-10T08:16:21.134 INFO:teuthology.orchestra.run.smithi155.stderr: 92.4% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.b.log.gz 2024-02-10T08:16:21.570 INFO:teuthology.orchestra.run.smithi070.stderr: 92.5% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.c.log.gz 2024-02-10T08:16:23.631 INFO:teuthology.orchestra.run.smithi070.stderr: 91.7% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-mon.a.log.gz 2024-02-10T08:16:28.548 INFO:teuthology.orchestra.run.smithi155.stderr: 93.9% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.6.log.gz 2024-02-10T08:16:28.613 INFO:teuthology.orchestra.run.smithi070.stderr: 93.9% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.2.log.gz 2024-02-10T08:16:28.633 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.5.log.gz 2024-02-10T08:16:29.224 INFO:teuthology.orchestra.run.smithi155.stderr: 93.9% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.7.log.gz 2024-02-10T08:16:29.318 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.3.log.gz 2024-02-10T08:16:29.436 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.1.log.gz 2024-02-10T08:16:29.568 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.4.log.gz 2024-02-10T08:16:29.570 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-02-10T08:16:29.570 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m9.790s 2024-02-10T08:16:29.570 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m36.819s 2024-02-10T08:16:29.570 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m1.312s 2024-02-10T08:16:29.882 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/70142e68-c7e6-11ee-95b8-87774f69a715/ceph-osd.0.log.gz 2024-02-10T08:16:29.884 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-02-10T08:16:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m10.107s 2024-02-10T08:16:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m42.605s 2024-02-10T08:16:29.884 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m1.569s 2024-02-10T08:16:29.885 INFO:tasks.cephadm:Archiving logs... 2024-02-10T08:16:29.885 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217/remote/smithi070/log 2024-02-10T08:16:29.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T08:16:35.722 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217/remote/smithi155/log 2024-02-10T08:16:35.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T08:16:41.132 INFO:tasks.cephadm:Removing cluster... 2024-02-10T08:16:41.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 --force 2024-02-10T08:16:41.324 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T08:16:41.626 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 70142e68-c7e6-11ee-95b8-87774f69a715 --force 2024-02-10T08:16:41.819 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 70142e68-c7e6-11ee-95b8-87774f69a715 2024-02-10T08:16:42.127 INFO:tasks.cephadm:Removing cephadm ... 2024-02-10T08:16:42.128 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T08:16:42.145 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T08:16:42.161 INFO:tasks.cephadm:Teardown complete 2024-02-10T08:16:42.162 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-10T08:16:42.173 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-10T08:16:42.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-10T08:16:42.188 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-10T08:16:42.272 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-10T08:16:42.281 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-10T08:16:42.281 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T08:16:42.283 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T08:16:42.295 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-02-10T08:16:42.297 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-02-10T08:16:42.300 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T08:16:42.300 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-02-10T08:16:42.300 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +457us[ +457us] +/- 85ms 2024-02-10T08:16:42.300 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 7 377 17 -98us[ -115us] +/- 65ms 2024-02-10T08:16:42.300 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -354us[ -354us] +/- 89ms 2024-02-10T08:16:42.300 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T08:16:42.301 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T08:16:42.301 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-02-10T08:16:42.301 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +473us[ +473us] +/- 85ms 2024-02-10T08:16:42.301 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -116us[ -124us] +/- 65ms 2024-02-10T08:16:42.301 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -347us[ -355us] +/- 89ms 2024-02-10T08:16:42.301 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T08:16:42.302 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-10T08:16:42.311 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-10T08:16:42.312 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-10T08:16:42.320 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T08:16:42.416 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T08:16:42.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T08:16:42.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:16:42.521 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T08:16:42.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:16:42.550 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-10T08:16:42.560 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-10T08:16:42.568 INFO:teuthology.task.internal:Duration was 3123.970879 seconds 2024-02-10T08:16:42.568 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-10T08:16:42.577 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-10T08:16:42.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T08:16:42.580 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T08:16:42.620 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T08:16:42.634 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T08:16:43.046 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-10T08:16:43.046 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-02-10T08:16:43.046 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T08:16:43.092 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-02-10T08:16:43.092 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T08:16:43.119 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-10T08:16:43.119 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T08:16:43.134 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T08:16:43.187 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-10T08:16:43.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T08:16:43.202 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T08:16:43.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-10T08:16:43.658 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-10T08:16:43.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T08:16:43.685 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T08:16:43.713 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-10T08:16:43.724 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:16:43.728 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:16:43.750 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-02-10T08:16:43.779 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-02-10T08:16:43.814 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:16:43.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:16:43.829 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:16:43.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:16:43.869 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-10T08:16:43.879 INFO:teuthology.task.internal:Transferring archived files... 2024-02-10T08:16:43.880 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217/remote/smithi070 2024-02-10T08:16:43.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T08:16:43.945 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217/remote/smithi155 2024-02-10T08:16:43.945 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T08:16:43.984 INFO:teuthology.task.internal:Removing archive directory... 2024-02-10T08:16:43.984 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T08:16:43.986 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T08:16:44.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-10T08:16:44.043 INFO:teuthology.task.internal:Not uploading archives. 2024-02-10T08:16:44.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-10T08:16:44.071 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-10T08:16:44.071 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T08:16:44.073 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T08:16:44.085 INFO:teuthology.orchestra.run.smithi070.stdout: 265715 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 08:16 /home/ubuntu/cephtest 2024-02-10T08:16:44.088 INFO:teuthology.orchestra.run.smithi155.stdout: 265832 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 08:16 /home/ubuntu/cephtest 2024-02-10T08:16:44.089 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-10T08:16:44.100 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-10T08:16:44.120 INFO:teuthology.nuke:Checking targets against current locks 2024-02-10T08:16:44.144 DEBUG:teuthology.nuke:shortname: smithi070 2024-02-10T08:16:44.145 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T08:16:44.171 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:16:26.337229', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6t5GCr8qlR/nF+e2IdSDCRLhpYRjYi9g1KBMBI9nehO22Aod6MfVunPPjNGq/ORhO4eH+cNSTIvWQJei7iSAo='} 2024-02-10T08:16:44.174 DEBUG:teuthology.nuke:shortname: smithi155 2024-02-10T08:16:44.174 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T08:16:44.201 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555217', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:16:26.339367', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNwKW7o/izMnyH1Q74VDiFWfz/BCGC7DXFHyfUX0tcoJpSrui5C1nhVj4u1Ce7yT2gQTde7CdQrlh82g6pQxwOw='} 2024-02-10T08:16:44.230 INFO:teuthology.orchestra.console.smithi070:Power off 2024-02-10T08:16:44.231 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T08:16:44.255 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2024-02-10T08:16:44.256 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:16:44.272 INFO:teuthology.orchestra.console.smithi155:Power off 2024-02-10T08:16:44.273 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T08:16:44.283 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-02-10T08:16:44.296 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-02-10T08:16:44.297 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:16:44.318 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-02-10T08:16:48.284 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:16:48.320 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:16:48.394 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-02-10T08:16:48.429 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-02-10T08:16:52.396 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:16:52.429 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:16:52.506 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2024-02-10T08:16:52.540 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-02-10T08:16:52.608 INFO:teuthology.orchestra.console.smithi070:Power off completed 2024-02-10T08:16:56.544 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:16:56.655 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-02-10T08:16:56.755 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-02-10T08:16:56.860 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_api_tests} duration: 3123.97087931633 failure_reason: '"2024-02-10T07:50:00.000131+0000 mon.a (mon.0) 2853 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-02-10T08:16:56.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T08:16:56.946 INFO:teuthology.run:FAIL